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ABSTRACT 



This report examines the nature and extent of professional 
development activities, assesses the current National Center for Education 
Statistics (NCES) collection of professional development data, and recommends 
ways to enhance that data collection. Section 1, "Overview and Background," 
introduces the subject. Section 2, "Linking Student Learning, Teaching 
Quality, and Professional Development," describes linkages between the three. 
Section 3, "Professional Development Indicators," outlines a taxonomy of 
professional development indicators of effectiveness in light of recent 
research and thinking. Section 4, "Measuring Professional Development Process 
and Outcomes," describes customary approaches to measuring professional 
development, focusing on teacher surveys, teacher interviews, observations of 
teaching practice, observations of professional development activity, and 
analysis of student achievement test results. Section 5, "Professional 
Development as Measured through Existing Surveys, " analyzes the focus and 
usefulness of available survey items with respect to inclusion in NCES data 
collection activities. The survey items include professional development 
design, delivery, content, context, and outcomes. Section 6, "Recommendations 
for NCES Data Collection on Professional Development," recommends new items, 
instruments, and methods that will maintain important time series yet 
generate more comprehensive data sets for supporting characterizations of 
practice more responsive to present conceptions about educational outcomes. 
Section 7 offers conclusions. The four appendixes include referenced 
background materials and items from 22 surveys used by NCES and others to 
learn about professional development. Appendices contain, "The National 
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I. Overview and Background 



Critics of public education are divided on many issues, but agree on one: too few children 
leave school with the knowledge and skills that their credentials were intended to certify. As a result 
of widespread dissatisfaction with this outcome, thoughtful research and documentation of successful 
practice have fostered many promising reform movements . New theories of instruction growing out 
of new understandings of child development, learning and teaching styles, and organizational 
structures have spawned classrooms, schools, and districts that are heavily involved in change and 
experimentation. Many such movements are small, sometimes involving only a handful of teachers or 
a school or two; others are large and growing. Despite differences in size or intensity, most 
transformation efforts depend on improvements in teaching quality, typically brought about by 
program-related professional education, as one engine driving school and classroom change. Other 
resources may be brought to bear; curriculum, facilities and equipment, organizational arrangements, 
and family and community engagement all have roles to play. Increasingly, however, professional 
development leading to improvement in teaching quality is a key component of reform. 

As the twentieth century comes to a close, schools face both a need to change and 
opportunities to succeed. Many who are deeply concerned about public education believe that 
professional development for teachers, effectively conceived and delivered and aligned with other 
dimensions of the education enterprise, can be a primary support for such reforms. In 1994, this 
conviction led to the addition of professional development to the list of National Goals: 

By the year 2000, the Nation’s teaching force will have access to programs for the continued 
improvement of their professional skills and the opportunity to acquire the knowledge and 
skills needed to instruct and prepare all American students for the next century. 

The objectives for this National Goal address the need to engage states, school districts, and 
community partners collaboratively in recruiting, preparing, and maintainin g a hi ghl y qualified 
education workforce equipped to enable diverse student populations to achieve c hall eng ing academic 
goals. (See Appendix A.) Ten indicators were reco mm ended for inclusion in the 1995 National 
Education Goals Panel Report (National Education Goals Panel, 1995). 

Clearly, one direct way to enhance teacher quality is by revamping teachers’ preservice 
experiences, but even if that process were ideal, it would be insufficient. New teachers constitute a 
small segment of the nation’s education team. In the last three years, only one teacher in ten 
currently in the workforce had received intense pedagogical instruction such as that now offered in 



preservice teacher education programs (NCES, 1995). Instead, the majority of classroom teachers are 
far and getting farther from such training: teachers are older and more experienced than they were 
30 years ago. In 1966, the median age of all public school teachers was 36, and their median 
experience was eight years; in 1991, the median age of teachers was 42 and the median experience 
was 15 years (NCES, 1995, Table 68, p. 79). That aging trend is continuing. In 1993-94, 65 
percent of public school teachers had ten or more years of experience, and 30 percent had 20 or more 
years (NCES, 1995). While experience is a definite asset in professional practice, it may also signal 
less recent exposure to extended professional education and less familiarity with new approaches. 

The 30 percent who are 20-year teaching veterans, for example, may have received their last intensive 
professional education in the 1960s, when much less was known about teaching and learning. 

Spurred by the need for a maturing workforce to keep current with the expanding professional 
knowledge base, emboldened by a commitment to help every child achieve high standards, and guided 
by new conceptions of how reforms occur, school communities are increasing attention to professional 
development as a central component of reform. 

Like other aspects of schooling, professional development itself has changed as the result of 
an improvement in our understanding of and reliance on new models of teaching and learning. Until 
recently, professional development often consisted of single events designed to transmit new 
knowledge or well-defined skills, such as those needed to interpret test scores or complete a specific 
type of lesson plan. Although these efforts might have been related to some larger plan, they were 
not necessarily linked to the participants’ goals for improving teaching and learning in any systematic 
way. Implementation was thus often flawed or limited, and desired effects were hard to document. 
New approaches provide a wide range of opportunities for teachers and others to explore rigorous and 
relevant ideas about child development, learning, and teaching based on current research and proven 
practices. Teachers’ competence may be enhanced by creating and evaluating curriculum, designing 
assessments, participating in decision-making, joining networks of colleagues with similar subject 
matter or pedagogical interests, or any of a myriad of practice-related activities, in addition to 
traditional workshops and courses. 

In 1995, the U.S. Department of Education (ED) created the Professional Development Team 
to assist policymakers and practitioners in making maximum use of the best ideas about professional 
development as the cornerstones of their reform efforts. According to this team, the mission of 
professional development is to "prepare and support educators to help all students achieve to high 
standards of learning and development" (ED, 1996). Its principles assert that effective professional 
development: 






Focuses on teachers as central to student learning 



• Reflects best available research and practice 

• Promotes continuous inquiry and improvement 

• Is best evaluated ultimately by its impact on teacher effectiveness and student learning 

• Is planned collaboratively 

The team’s full statement of mission and principles is included in Appendix B. 

Similarly, the American Federation of Teachers (AFT) characterizes professional development 
as a "continuous process... [that] should empower individual educators and communities of educators 
to make complex decisions, identify and solve problems, and connect theory, practice, and student 
outcomes" (AFT, 1995). Its guidelines define sound professional development as that which: 

• Ensures depth of content knowledge 

• Provides a strong foundation in content pedagogy 

• Provides more general knowledge about teaching and learning processes 

• Takes a variety of forms 

The complete AFT guidelines are included in Appendix C. 

Although many view professional development as centrally important to school improvement, 
comprehensive assessment of such learning opportunities has yet to be undertaken. The National 
Center for Education Statistics (NCES), primarily through the Schools and Staffing Survey (SASS), 
collects some data, yet the National Education Goals Panel (NEGP) found little information on which 
to base its yearly evaluation of progress toward the new national goal. Indeed, examining the breadth 
and depth of information on professional development collected by a number of sources reveals 
overlaps and gaps. While much general information distinguishes among the professional 
development delivery types, little is collected on the objectives and design of those experiences or on 
other processes that stimulate teacher learning, such as action research or participating on a school 
improvement team. Items from SASS are the most extensive, while information from the National 
Assessment of Educational Progress (NAEP) and National Education Longitudinal Study of 1988 
(NELS:88) focuses primarily on measuring contact hours by content area. 

The purpose of this report is to lay the groundwork for building NCES capacity to assess the 
nature and extent of professional development activities as an increasingly vital element of the nation’s 



education systems. The report is organized into six sections and four appendices. Section II 
describes the linkages between student learning, teaching quality, and professional development, while 
Section HI outlines a taxonomy of professional development indicators of effectiveness in light of 
recent research and thinking. Section IV describes customary approaches to measuring professional 
development, and Section V analyzes the focus and usefulness of available survey items with respect 
to inclusion in NCES data collection activities. In Section VI we recommend new items, instruments, 
and methods that will maintain important time series, yet generate more comprehensive data sets that 
will support characterizations of practice more responsive to present conceptions about educational 
outcomes. Section VH presents our conclusions. The appendices include referenced background 
materials and items from 22 surveys now used by NCES and others to learn about professional 
development. 

In this report, we examine the nature and importance of professional development, assess the 
current NCES collection of important professional development data, and recommend ways to 
enhance that data collection. The culmination of the activities initiated with this paper will be 
development of further capacity to assess how the nation’s school systems are creating opportunities 
for teachers to learn, what skills and knowledge they target for teacher learning, what purposes they 
aim to serve by this investment, and what effects they achieve. We expect that the information 
collected with this expanded capacity will add to the definition and usefulness of the professional 
development data already collected and analyzed by NCES and others . 



II. I inking Student Learning, Teaching Quality, 
and Professional Development 



This section explores the rationale for providing teachers with professional development that 
enhances student learning. First, we describe student learning in terms of the content and processes 
of instruction. Second, we explain how teaching quality shapes learning experiences for students. 
Then we explore several ways that teaching quality may be influenced by professional development 
and show how conceptions of learning opportunities for teachers have changed to reflect new 
understanding of the acquisition and use of professional expertise. Finally, drawing on a review of 
recent research on this subject, we propose a set of indicators of high quality in five dimensions of 
professional development: design, delivery, content, context, and outcomes. 



Student Learning 

The primary goal of education is student leaming-the acquisition of knowledge, skills, and 
attitudes. Learning is the result of substantive student work directly related to lesson objectives, and 
it occurs when the complexity of instruction is well-matched to the student’s level of competence, 
motivation, and task engagement at the time of the lesson. Increased student achievement is the 
ultimate evidence of school success. At the heart of all school activity is a single, pivotal point: 
productive student engagement in a purposeful, academic task. Other aspects of desired student 
growth— for instance, citizenship and responsibility-are usually taught in the context of academic 
work. 



Assumptions about the process of learning shape assumptions about teaching effectiveness, 
and both sets of assumptions have recently evolved at a rapid pace. According to Resnick (1989), 
contemporary cognitive theory highlights three dimensions of learning that influence conceptions of 
teaching. First, learning involves interpreting information, not simply recording it. Second, new 
learning builds on prior lear ning ; the previously unknown must find a niche among the known if it is 
to be acquired at all. Third, new learning is acquired in context; context provides clues about 
meaning that strengthen acquisition and ownership. Research on teaching that accommodates this 
notion of learning is far from complete; however, while future refinements of what it really means to 
know something may change the terms of discourse about student learning, it will not alter the need 
to keep the quantity and quality of student engagement as the focus of improvement efforts. New 
ideas about learning may suggest additional strategies that will eventually prove or disprove their 



merit, but student engagement in work aimed at achieving a meaningful goal will always be central to 
the process. The review that follows draws on examples of student learning and sound instructional 
practices from a range of literature that bears on current conceptions of learning. 



Academic Learning Time 

In a study of teacher activities and environmental factors that influence student learning 1 
(Fisher, Berliner, Filby, Marliave, Cahen, & Dishaw, 1980), researchers traced the origins of 
achievement to the amount of time students spend actively engaged in a certain kind of work. They 
called this "academic learning time" (ALT). The study was narrow in its focus— basic skills in third 
and fifth grade reading and math— but broadly encompassing in its conceptual frame. Although recent 
studies have pushed the boundaries of our understanding of the content and processes of learning in 
important dimensions, the concept of ALT permits us to orient these new ideas to our central 
concern. Intentional assessment— formal or informal, "authentic" or otherwise-provides the measure 
of learning after it happens; in contrast, ALT offers immediate and observable evidence that the 
teacher is providing usable opportunities for students to learn. While time spent on other school 
activities may have value, ALT contributes most directly to achievement. 

ALT has two defining qualities: 

• Student work is directly related to academic goals 

• The work’s level of difficulty is well-matched to the student’s present level of 
competence 

ALT is distinguished from "engaged time" by these quality indicators. Students may be engaged for 
long periods of time without learning much; most of what students and teachers both see as "busy 
work" is engaging without being thought-provoking or generative. Likewise, ALT is distinguished 
from allocated time, which is simply the amount of time scheduled for a lesson. Wi thin limi ts 
sketched by contextual variables— for example, the development stage of the learners— the more ALT 
in a student’s school day, the more learning. 



1 The Beginning Teacher Evaluation Study, funded by the National Institute of Education, aimpH initially to 
identify the knowledge and skills required to begin teaching successfully. However, it developed into a study of 
the nature of instruction that generated a need first to characterize student learning. Its description of a learning 
event, perhaps dated in some ways, remains conceptually cogent. 



Resource shortages such as limited computer availability, lack of textbooks, and deteriorating 
physical plants are academically problematic because they impair the efficiency of student work. 

Many problems that should be solved for the general welfare of children and their families also put 
academic success at risk because they constrain students’ engagement in schoolwork. To tie student 
learning to student work is not to hold the student solely accountable for learning or to discount the 
seriousness of factors that interfere with student work (and cause other problems as well); rather, it is 
to recognize that from an educational perspective the capacity of these factors to impede student work 
renders them worthy of attention. Limited investment of personal effort in learning, whatever the 
cause, results in limited learning. 



Substantive Work 



Substantive work directly and appropriately related to the desired outcome is an essential 
component of academic learning time. In an example suggested by the study of Fisher et al. (1980), 
if the fifth-grade lesson’s goal is learning to solve "story problems," then substantively appropriate 
work presents effective solution strategies and involves the student in real solution activities, not just 
in routinely adding (subtracting, dividing, etc.) the two numbers buried in the narrative. Within the 
cycle of lear nin g and assessment, ALT involves work on activities that cultivate capacity to meet the 
learning goal and satisfy the assessment standard. While in some ways self-evident, this point is 
important because it distinguishes a particular kind of classroom work from many other common 
classroom activities that may also be engaging and time consuming. Setting up and taking down 
equipment, copying assignments, and coloring worksheets may be enjoyable and/or necessary 
activities, but they do not contribute to achievement in the direct and efficient way that cognitively 
demanding tasks relevant to academic goals do. 

In a recent report about authentic pedagogy (Newmann, Marks, & Gamoran, 1995), a team of 
researchers described a more detailed notion of productive academic work and showed its contribution 
to student achievement. "Authentic academic performance," in their view, engaged students in 
analytic thinking, demonstration of important disciplinary concepts, and elaborated communication. 

By analytic thinkin g, they meant such cognitive activities as "organizing, synthesizing, interpreting, 
hypothesizing, describing patterns, making models or simulations, constructing... arguments or 
inventing procedures." They construed engagement with important disciplinary concepts as not just 
applying rules, but extending definitions and making connections with other concepts within the 
discipline or to another discipline. By elaborated communication, they meant conveying information 
or ideas concisely, logically, and with appropriate explanation or argument. Where the researchers 
found evidence that students were more engaged in these ways, they also found superior student 



achievement on assessments designed to measure this kind of outcome. In schools where advocacy of 
and support for such activities was strong overall but implementation was uneven across classrooms, 
classrooms scoring high in "authentic learning" activities were also the high scorers on achievement 
measures. 

Despite the focus on different subjects, the similarity of findings in both Fisher et al. and 
Newmann et al. suggests the robustness of the notion that the quality of student work in substantive 
dimensions generally under the teacher’s control is a primary contributor to students’ achievement. 



Matching Work to Learner 

Task engagement is most productive when the goal is just beyond the reach of the student at 
the beginning of the work period, yet able to be mastered by the end of it. Initial studies of learning 
as a function of the level of difficulty of the learning task suggested that generally students learn more 
when they experience high rates of success in their academic work (Fisher et al., 1980), especially in 
their independent work. Further analyses showed that the optimal rate of success is determined by 
the interaction of the student and the learning task. On the one hand, if the goal is to master a skill to 
the point of automaticity, for example in math facts or spelling, then students may learn most 
efficiently when practice activities can be completed with high levels of accuracy. On the other hand, 
if the goal is more complex, for example to learn more about causal relations in natural science or 
characterization in literature, then learning activities may usefully be more challenging. Furthermore, 
some students will persevere on very difficult tasks with ultimate success, while others may fail to 
grasp the elusive objective if it is too distant from what they already know. Finally, task engagement 
is influenced by the scaffolding of the event, that is, the amount of structure, explanation, and explicit 
direction offered to learners. Students who begin the lesson with different levels of skill and 
knowledge need to receive different support, provided to ensure optimal cognitive stretch to each 
without undue frustration. 

Doyle captured students’ experience of a lesson in his notion of academic task structures, 
which he called "the central organizing frameworks in classroom settings" (1992, p. 503). Academic 
tasks shape the way cognition is elicited in a learning event, whether it is a simple activity completed 
in one session or a complex activity spanning an extended time period. Each task is defined by its 
intended product, the kind of work needed to produce the product, the resources available to support 
the work, and the importance assigned to the product in the relevant accountability system (Doyle, 
1986). These features communicate to the learner what is expected and how a given expectation is 
prioritized in the learning system (and therefore how much is to be invested in meeting expectations). 



Academic tasks are a kind of "molecule" of learning— the smallest unit that engages students 
meaningfully. 

Students experience ALT in the form of academic tasks. Analyses of observations of students 
at work on different kinds of tasks (cited in Doyle, 1992) revealed that familiar, routine tasks support 
order and the quick flow of work. While order and pacing are valuable, the content of familiar, 
routine activities is necessarily limited in scope and challenge. On the other hand, problem-centered 
tasks that engage students in interpreting and making decisions proceed more slowly and elicit anxiety 
in some students. These students may respond by soliciting more structure or disengaging from the 
task. The instructional goal is to maintain reasonable order while stimulating growth. When student 
efforts focus on academic tasks that have the right blend of substantive content and level of challenge, 
then learning is the result. Conversely, when substantive content is inadequate or the level of 
challenge is inappropriate, effort will be less fruitful. 



Teaching Quality That Increases Student Learning 

Teaching quality may be demonstrated by the capacity to generate and sustain effective 
student engagement. Teaching quality impinges directly on the efficacy of student work through 
professional competence in two areas: 



• Teachers’ content expertise, expressed as knowledge of a specific discipline and 
knowledge of how to teach that discipline 

• Teachers’ pedagogical expertise, expressed as ability to develop academic tasks with 
appropriate content and challenge, use a variety of instructional strategies, assess 
students’ knowledge and skill, and manage the classroom environment for optimal 
working conditions 



Content Expertise 

Teachers’ understanding of lesson content, their repertoire of strategies suitable for making 
particular content accessible to a given group of students, and their understanding of what students 
already know about that content influence the success of a learning activity. 

First, teachers’ knowledge of specific content is the foundation for setting up academic tasks. 
Whether students are listening to a lecture, puzzling out a discovery exercise, working with 
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manipulatives, or acting out a scenario, the quality of the activity’s content hinges to some degree on 
the teacher’s understanding of the content. If the teacher’s knowledge of the content is shallow or ill- 
formed, the teacher’s instruction will reflect that limitation, for example by presenting erroneous 
characterizations or only the most trivial instances rather than those that truly illuminate a concept. 
Some studies reveal that teachers teaching out of their fields of expertise rely more heavily on drill 
and practice activities and entertain student questions less willingly than those who were prepared to 
teach the same subjects and thereby are able to navigate the sometimes unpredictable turns of a 
complicated lesson (Grossman, 1991). The explosion of knowledge in an information age makes it 
unlikely that even the most dedicated scholar could claim mastery over an entire discipline, and the 
ascendance of the model of teacher as guide suggests that dispensing information from one’s personal 
store of knowledge is no longer a sufficient conception of teaching. However, understanding what 
Schwab called the "structure of a discipline" (1964) and how knowledge is construed and developed 
within a discipline remains essential. Knowing the terms and procedures of discourse enables a 
teacher to model for students the way learning is accomplished and expanded in a discipline. 

Second, teachers’ knowledge of how to teach that content— what. Shulman (1987) and his 
colleagues call pedagogical content knowledge— supports insightful selection of explanatory 
frameworks in consideration of the content and the students. This form of expertise enables a teacher 
to choose the examples, metaphors, or applications of a concept or skill that make the content most 
accessible to a given group of students. Teachers with limited knowledge of a discipline may choose 
a handy example that only appears to illustrate the concept but doesn’t hold up under scrutiny. They 
may have so limited a collection of instances that students generalize to the wrong conclusion or 
acquire a mistaken notion of the lesson’s point. In contrast, teachers with pedagogical content 
knowledge present "the most useful forms of representation of those ideas, the most powerful 
analogies, illustrations, examples, explanations, and demonstrations— in a word, the ways of 
formulating the subject that make it comprehensible to others" (Grossman, 1991, paraphrasing 
Shulman). Such teachers flag the key ideas and defining principles of a discipline and are not easily 
distracted by issues at the margins. 

This dimension of expertise overlaps with a third dim ension: the capacity to assess and 
recognize the significance of what students already know in relation to what could be known about a 
topic. This capacity enables teachers to choose the right point of entry for lessons and the right 
lear nin g goals for a given period of work. This dimension of content expertise helps the teacher 
compose a lesson framed to offer the most helpful and authentic view of the broader subject, a view 
that takes into account what the students can already "see" of the subject and helps them perceive new 
relationships and patterns. Knowledgeable teachers directly address the points that their experience 



has taught are problematic to certain types of students and link new content to that which is already 
familiar to students. 



Pedagogical Expertise 

We use the term "pedagogical expertise" to refer to general teaching skills and knowledge, 
distinguished from those discussed above that are related primarily to specific subject-matter content. 
Pedagogical expertise includes the teacher’s understanding and application of generally accepted 
principles of instruction, classroom management, and assessment of student learning. 

First, a teacher’s overall instructional skill is the basis on which subject-specific refinements 
are added. Process/product research, which examines the correlations between observable 
instructional factors and student learning, has established that certain teacher-controlled instructional 
conditions are reliably associated with student learning. A summary of this research (Brophy & 

Good, 1986) lists six factors that promote learning: 

• Content coverage is extensive and the pace of instruction is brisk 

• Content presentation is organized, clear, and appropriately redundant 

• Questioning varies with respect to level of difficulty and cognitive complexity, in 
keeping with students’ interest and engagement and the subject’s nature 

• Teacher reactions to students’ contributions encourage participation without 
discouraging excellence 

• Independent homework and seatwork assignments engage students at high rates of 
successful practice 

• Practices with respect to individual and group work sustain productive engagement 
and make the best use of teacher resources 

Teacher competence in any of these areas is a matter of knowledge, skill, and judgment. For 
example, research and simple logic suggest that students are more likely to learn what has been taught 
than what has not been taught, so teachers who cover more material establish the basis for increased 
learning. Racing from topic to topic to get to Grover Cleveland by April in a history course or finish 
the targeted chapters of the math text before spring testing does not necessarily produce real learning; 
engagement with the content is also a critical ingredient of mastery. Skillful teachers are those who 



set a pace as brisk as students can manage, continuously adjusting it to ensure engagement and boost 
mastery while maintaining ambitious goals. Engagement and coverage both play roles. 

Second, classroom management skills make a direct contribution to the amount of time 
available for learning and to the attitudes and motivation with which students approach the task. 

Studies begun in 1970 by Kounin and continuing have identified the teacher classroom management 
skills most commonly associated with student achievement (Evertson, 1987). At the elementary level, 
students who learned more had teachers who: 

• Analyzed classroom tasks to understand the demands made on students 

• Taught students how to follow key rules and implement routine procedures 

• Viewed the classroom from the students’ perspective to understand what directions 
students needed to function successfully 

• Monitored student behavior closely and responded immediately to problems 

At the secondary level, researchers observed that successful students had teachers who, in addition to 
explaining rules and monitoring behavior, also: 

• Implemented effective accountability systems to keep track of students’ work 

• Communicated information and expectations clearly 

• Organized instruction to get the most time for academic work 

The specific ways that good teachers implement strategies directed toward establishing and 
maintaining civility and good work habits vary, but their effectiveness in this aspect of teaching, 
uniformly and directly influences how much time is available for students to work. 

Third, teacher’s skill in assessment also contributes to student learning. In Doyle’s concept of 
an academic task (1992), the type and importance of assessment communicates to students which task 
dimensions are essential and how the task figures in the overall scheme of the course or lessons, and 
what resource allocations (e.g., students’ time) are warranted. Students’ motivation and learning is 
directly affected by the teacher’s ability to design assessment strategies that target instructional goals 
precisely, capture evidence of learning effectively, and integrate the assessment into a sensible 
accountability system. 



In their studies of "authentic pedagogy," Newmann, Marks, and Gamoran (1995) describe 
specific assessment strategies associated with an approach to instruction that leads to higher student 
achievement. In their view, "authentic" assessment involves students in, first, organizing 
information; second, evaluating alternatives; third, demonstrating understanding of disciplinary 
content; fourth, demonstrating discipline-appropriate methods of inquiry; and fifth, communicating 
understanding in an elaborated written assignment. The assessment task itself should show the value 
of the knowledge being tested outside of school, by centering on a problem that has real-world 
applications and eliciting communication with an audience besides the teacher. 



Professional Development That Increases Teaching Quality 

Teachers must provide meaningful academic tasks that engage students in work directly 
related to learning goals, and support students in a way that will ensure their success with 
appropriately c hall enging content. However, accomplishing these goals depends on mastery of an 
® ever-expanding pedagogical knowledge base. Since the 1960’s, 1970’s, and even the 1980’s, when 

most of today’s teachers last studied their profession for an extended period, a major expansion has 
occurred in the education community’s understanding of how to frame lessons, how to engage 
students, and what the curriculum should include. New techniques for diagnosing and assessing 
£ students’ learning support more precise notions about how they may best be served in schools. 

Furthermore, the trend for more of the important instructional decisions to be made at the school or 
district level means that education systems rely on continuing education that targets specific programs 
or strategies to cultivate the skills and knowledge necessary to implement local decisions. Formal 
p preservice education can lead to acquisition of the appropriate general knowledge and skills, but 

building capacity to implement school progr ams relies on tailor-made professional development. In 
addition, adopting new approaches requires that teachers learn enough to make informed decisions 
and to implement those choices in their classrooms and schools— another target for professional 
^ development. 

Content expertise and pedagogical expertise are qualities that can be learned, and indeed they 
are focuses for preservice and continuing professional education. On the one hand, teachers, like 
other learners, gain expertise from practice. Their ongoing engagement in explaining content, 

• planning lessons, and responding to students’ questions deepens and probably extends their mastery of 

core professional skills and knowledge. For example, years of explaining the rules for comma usage 
will no doubt produce great adeptness in an English teacher’s competence in teaching that aspect of 
punctuation. On the other hand, because some new professional knowledge arises outside the arena 
£ of classroom practice, those whose experience is circumscribed by practice will be limited in their 
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professional learning. An English teacher’s successful implementation of a "writing workshop" 
approach is less likely to emerge spontaneously without deliberate study, because it is a complex 
strategy. Evidence from early, well-documented trial-and-error efforts is now available to show 
teachers how to ease adoption and enable them to elicit more immediate success for students. 
Professional development activities are designed to enhance practice directly by addressing the needs 
for expertise that emerge from a desire to boost instructional effectiveness. 

Professional development enhances teachers’ knowledge and skill, as well as their capacity to 
exercise professional judgment. Although some professional development programs, properly 
conceived and implemented, have succeeded in raising teaching quality, others have had little effect. 
This variation in outcomes suggests that all approaches to professional development are not created 
equal. Furthermore, while commonly held conceptions of what constitutes high-quality professional 
development have evolved over the last decade or so, in the minds of many educators, professional 
development is synonymous with training. Training activities include the workshops famili ar to all 
teachers, as well as various kinds of individual coaching or technical assistance services. In all these 
arrangements, a trainer—generally recognized as having special expertise-directs the content and flow 
of professional development activities. 

Well-designed training programs advance teachers’ knowledge of instructional practice and 
support them in developing new classroom skills. Research on the implementation of innovations and 
specific professional development ventures (Sparks, 1986; Showers, Joyce, & Bennet, 1987; Joyce & 
Showers, 1988; Smylie, 1988) has refined our understanding of how and under what conditions 
effective training programs work. Depending on the desired outcomes, tr ainin g might include 
exploration of theory, demonstration or modeling of a skill, practice of the skill under simulated 
conditions, feedback about performance, and coaching in the workplace (Joyce & Showers, 1988). 
Good training provides adequate opportunities for practice and for classroom consultation and 
coaching as teachers learn to apply new concepts. As teachers explore a new theory and knowledge 
base, they see these concepts applied to their own classroom situation, practice applying them as a 
coach observes, and receive feedback on an ongoing basis. Effective training supports teachers as 
they internalize a new practice and make decisions about how to use it to support student engagement 
in challenging academic tasks. 

Judith Warren Little (1993) argues that traditional training works reasonably well to introduce 
teachers to those aspects of reforms that are "technical, ” understood as an expansion of the existing 
repertoire of classroom practices. Training can help teachers acquire the ability to use a new 
computer software program or interpret test scores and prescribe follow-up instruction. However, the 
traditional training model has limited capacity to help teachers meet the demands of current reforms 



based on new conceptions of student learning, because adopting such reforms does not involve simply 
replicating proven behaviors in local settings; adoption involves making judicious changes that affect 
not only courses and lessons but the whole learning environment as well. . 

Similarly, the RAND Change Agent Study found that skill-specific training activities enabled 
teachers to implement new project methods and materials, but only in the context of the particular 
setting of the project. When the project ended, teachers stopped using the project methods and 
materials because they had never really learned how to adapt them to other settings. In contrast, 
various staff supports-resource personnel, outside consultants, project meetings, and teacher 
participation in project decisions— produced stronger long-term effects. The RAND study concluded 
that teacher training was less about technology transfer and more about ongoing program support by 
an organization. According to the study’s authors, sometimes the learning task for teachers is more 
like problem solving than like mastering "proven" procedures (McLaughlin & Marsh, 1978). 

The more recent literature on professional development (Little, 1993; Lieberman, 1995; 
Darling-Hammond & McLaughlin, 1995) proposes an expanded notion of what constitutes good 
learning opportunities for teachers. These authors contend that teachers’ work in schools must be 
structured to allow teachers to learn together from the teaching and learning process and to work 
collaboratively to apply the broad principles of reform to specific classroom contexts. High-quality 
professional development, they hold, enables teachers to develop habits of collaboration and problem- 
solving within the school organization. These newer conceptions of teacher opportunity to learn 
derive less from research on alternative models of professional development— a field still developing— 
and more from research and theory on the professionalization of teaching and the contexts in which 
teachers work. 

In this section, we s ummar ize the arguments for setting new teacher learning goals that are 
more compatible with new learning goals for students, then present considerations of design, delivery, 
content, context, and outcomes critical to all improvement efforts. On these elements we build the 
system of professional development indicators described in a subsequent section. 



New Learning Goals for Teachers 

Most present refor ms are not readily expressed in terms of specific, transferable skills and 
practices (Little, 1993). Rather, the reforms require that teachers grapple with translating broad 
principles (like the standards developed by the National Council of Teachers of Mathematics) into 
practice. Teachers must work with different concepts of valued student learning, but without specific 



prescriptions for how their practice must change in response. Professional development for reform no 
longer comprises transmission of knowledge, but study by teachers of the teaching and learning 
processes. 

Little (1993) predicts that professional development responding to teachers’ broader learning 
needs will look quite different from conventional training programs: 

This calls not for training, but for adequate "opportunity to learn" (and experiment, consult, 
or evaluate) embedded in the routine org aniz ation of teachers’ work day and work year. It 
requires the kinds of structures and cultures, both organizational and occupational, compatible 
with the image of "teacher as intellectual" ... rather than teacher as technician, (p. 133) 

This shift in professional development is a shift "from replication to reflection, in which 
practicing teachers focus less on the transfer of knowledge and strategy and more on analytical and 
reflective learning" (Dilworth & Imig, 1995). In this vision, professional development aims to 
support adoption of new approaches to curriculum and instruction by stimulating their critical thinking 
about students’ needs and the teaching process. This reflective approach sharpens teachers’ skills in 
problem solving, determining students’ needs, and conducting action research that is designed to 
develop new knowledge and skills related specifically to their schools and classrooms. This level of 
professional development offers teachers meaningful intellectual, social, and emotional engagement 
with ideas, with materials, and with colleagues both in and out of teaching. In this more evolved 
model of professional development, teachers’ prior knowledge and skill is seen as a resource for 
future learning, and the systems that influence the quality of their teaching are brought into alignment 
around the desired change. 



What are Effective Designs of Professional Development? 

Teachers do not stimulate student learning in isolation. Success for all students depends upon 
both teacher quality and school quality in areas that support students’ academic engagement. 

Effective professional development programs, predicated on the principles advanced above, begin with 
designs that link teacher learning with broader school improvement goals. The blueprint for teacher 
learning activities arises from a coherent school plan, based on sound assessment of student needs and 
shared goals. When that happens, professional development activities are guided by a long-term plan 
that helps ensure that teacher learning will be sustained by the school environment rather than eroded 
by continual friction from disparate organizational forces. With a clear, coherent strategic plan in 
place, schools avoid the common pitfall of staff development programs that end up fragmented or 
piecemeal, with no thought given to follow-up or to how the new technique fits in with those 



imp lemented in previous years. When they are crafted to support school improvement goals, 
professional development activities gain coherence and power by building upon those goals and 
reinforcing teachers’ on-going learning. 



While designs will vary according to local needs and conditions, many common elements will 
characterize the designs of some of the most promising programs. Good designs: 



• Focus on organizational as well as individual support: Individual aptitudes and 
interests play an appropriate role in some professional development choices. 

However, to some extent, individual choices ought to be made in light of improving 
collective capacity, to ensure that school resources invested in professional 
development lead to enhanced school effectiveness. 

• Accommodate the need for long-term learning: Some important changes in behavior 
or practice will require extended attention. Plans for professional development should 
assume the need for teacher experimentation and practice over time. 

• Promote continuous inquiry, embedded in the daily life of the school: Effective 
practice involves continuous improvement. A professional development plan should 
allow participants to critique their own practice and the total program, to permit 
ongoing assessment of the match between what is needed and what is offered. 

• Address teachers’ evolving needs: Individual and team growth combined with normal 
staff turnover leads to new demands for professional development. The design should 
accommodate differences in professional needs. 

• Reflect participant input: Each player in the education community has a unique and 
valuable perspective on the nature of any collective problem and the best way to solve 
it. In a situation where resources of time and other supports for professional 
development are finite, taking participants’ opinions into account provides an 
important basis for prioritizing options. 

• Include evaluation: The costs of professional development, both direct and indirect, 
are considerable. Reasonable accountability demands that some of the investment in 
professional development be used to determine whether it achieved the desired results. 

• Ens ure equitable access: Differences in available resources often result in a "poor- 
get-poorer" professional development system. That is, circumstances in the neediest 
schools and for teachers of disadvantaged or minority students do not often support 
professional development activities as well as those in the richest schools. 

Professional development designs should aim to compensate for these differences. 



The way that the design of professional development links to broader school issues and 
incorporates elements that accommodate individual and organizational characteristics makes a strong 
contribution to its ultimate effectiveness in bringing about improved teaching quality. 



How Can Professional Development be Delivered? 

Four dimensions of delivery are commonly cited as germane to its effectiveness. First is the 
need to offer multiple points of entry and options for participation. Second is the importance of 
including in this range new forms of learning, such as collaboration, school or curriculum 
development projects, and inquiry. Third is the extent and nature of the offerings-ensuring that they 
occur with the intensity and duration necessary to promote real learning. Fourth is the integration of 
resources for learning available in many sectors of the professional community, making appropriate 
use of university faculty, central office experts, consultants, teacher leaders, and others. 

Multiple ways to participate. The formerly prevalent view of learning as knowledge 
transmission supported formal professional development events-a university course, a district 
workshop, a professional association training session-offered in educational settings away from the 
teaching workplace. Sessions emphasized efficient presentation of information, with little attention to 
ongoing support of continuous learning and changed practices. The expanded vision of professional 
development includes multiple forms of learning, many of which are job-embedded (Sparks, 1995). 
This shift in the notion of where professional development can or should take place signals a 
reconception of teacher learning, expanding beyond the transmission of knowledge and skills to 
teachers’ study of the teaching and learning processes. Teachers’ knowledge and understanding may 
grow through involvement in decisions about the substance, the process, and the organization of 
support for learning in school, as well as through inquiry. 

Lieberman (1995) comments that teachers must have opportunities to discuss, think about, try 
out, and hone new practices, suggesting that these opportunities to learn should be embedded in 
teachers’ and schools’ daily activities. Expanded professional learning opportunities for teachers 
require that schools: 

• Build new roles for teachers: teacher leader, peer coach, teacher researcher 

• Create new structures in which teachers can work together: problem-solving groups, 
decision-making teams 



• Invite teachers to work on new tasks: journals and proposal writing, learning about 
assessment, writing or analyzing case studies, creating standards 

• Create a culture of inquiry, wherein professional learning is expected, sought after, 
and an ongoing part of teaching and school life 

The life span of these activities is no longer one or two days; they become an integral part of 
teachers’ roles and an ongoing feature of the school. 

New forms of learning. Although training and the independent search for knowledge 
motivated by individual questions or circumstances will continue to play a role in professional 
development, the importance of other forms of learning will also increase. Many aspects of new 
learning are simply matters of skill-how to use a new software program or apply certain kinds of 
rubrics— and tr ainin g is an efficient way to acquire such skills. Other questions will be mostly of 
personal interest or relevance, and, like other professionals, teachers will search for answers in trade 
publications and other resources in a form of study sometimes called "individually guided 
development." Other models of learning experiences have begun to produce positive effects on 
teaching quality in enough instances to warrant optimism about their inclusion in the set of commonly 
used delivery systems for professional development. These include: collaboration through coaching, 
networking, and teaming to solve problems of practice; participating in school improvement and 
curricul um development activities; and conducting more formal inquiry, individually or with partners. 

Collaboration. Coaching, networking, and teaming have generated improvements in teaching 
quality in many settings. Peer coaching and critiques may play a key role in learning new 
approaches. Good training provides adequate opportunities for practice and for classroom 
consultation and coaching as teachers learn to apply new concepts. As teachers explore a new theory 
and knowledge base, they see these concepts applied to their own classroom situation, practice 
applying them as a coach observes, and receive feedback on an ongoing basis. This approach 
supports teachers as they inter naliz e a new practice and make decisions about how to use it to support 
student engagement in challenging academic tasks. 

One professional development strategy that several recent projects have found useful for 
maintainin g growth is the formation of teacher networks and subject-matter collaboratives. These 
inf ormal arrangements draw together teachers interested in a particular problem or approach to 
teaching. Networks are usually created to deal with complex educational problems that defy 
simplistic solutions and pat answers— the teaching of writing, for example. Networks provide access 
to new ideas and a supportive community in which to begin translating these ideas into meaningful 
action in schools and classrooms. Shared activities and experiences not only encourage teacher 



learning but also serve as organizing tools to keep teachers working together, sharing, and learning 
from one another over time (Lieberman & McLaughlin, 1992). 

School improvement and curriculum development. Developing curriculum, experimenting 
with new assessments, writing student performance standards, and participating in systematic school- 
improvement processes can be valuable opportunities to learn. Adults learn most effectively when 
they have a problem to solve, and teachers’ experiences may provide direction in framing problems 
and recognizing and/or developing relevant solutions (Sparks & Loucks-Horsley, 1990). Teachers 
also learn through such diverse means as conducting action research, participating in study groups or 
small-group problem solving, observing peers, keeping journals, and becoming involved in 
improvement processes. 

Inquiry. The delivery of good professional development often emphasizes inquiry that 
involves teachers in formulating questions about their own practice and systematically pursuing 
objective answers to those questions. This approach recognizes and nurtures teachers’ intellectual and 
leadership capacity. The overarching assumption is that: 

The most effective avenue for professional development is cooperative study by teachers into 
problems and issues arising from their attempts to make their practice consistent with their 
educational values. . . . [The approach] aims to give greater control over what is to count as 
valid educational knowledge to teachers (Ingvarson, cited in Sparks & Loucks-Horsley, 1990). 

Teacher inquiry may take place formally or informally, inside or outside the school, with pairs or 
groups of teachers, but its defining quality is its recognition of the rules of evidence that govern a 
given area of investigation. One of the best articulated examples of teacher inquiry is action research. 
With the assistance of an advisor often based at a university, teachers formulate research questions, 
gather and analyze data, and use their findings to improve classroom instruction. 

Duration and intensity. When the goal of professional education is implementation, its 
delivery should provide for participation over an extended time, so that teachers can explore new 
theories and practices, assess their meaning, and integrate them into their own repertoire. With rare 
exceptions, single events or "one-shot" workshops have little effect on teaching practice. Teachers 
who encounter a new idea or practice only once may gain awareness but seldom gain sufficient 
knowledge to change what they do in classrooms day in and day out. Professional development 
activities of all kinds are more powerful when they are coordinated so that understanding and skills 
can be built over time, reinforced continuously, and practiced in a variety of situations. To begin 
making the scope of changes envisioned by most reform movements, teachers clearly need more time 



for learning than the three or four days each year devoted to professional development on most 
district calendars. Many engaging professional development activities require teachers to meet several 
times a month over the course of an entire school year. The measure of adequacy in intensity or 
duration of a learning activity is evidence of change in the targeted dimension of practice. Among the 
most common explanations for the absence of change is insufficient time for learning. 

Integrating community resources. The current discourse about improving the education 
system highlights professionalizing teaching, reducing the "theory/practice gap" perceived between 
colleges of education and K-12 schools, and generating a more synergistic collaboration framework 
among those in different sectors of the educational community. University faculty, central office staff 
developers, school-based master teachers, consultants from professional associations, and others have 
contributions to make to teaching quality (Imig, personal communication, 5/14/96). The way they are 
used influences both cost and quality of activities. 



What is the Appropriate Content of Professional Development? 

Subject matter. Teaching to more rigorous standards embodied in new subject-specific 
content frameworks requires increased emphasis on content knowledge and content-specific 
pedagogical skills (AFT, 1995; Sparks, 1995). Concerns about standards in the broader community 
necessitate attention to new conceptions of what students should know and be able to do. Whether 
such standards are set locally or at some other level, they establish benchmarks for student learning, 
and therefore set expectations for teacher knowledge. Many teachers would benefit from professional 
development activities that acknowledge teachers’ prior content knowledge and skill, and build on that 
base to extend and improve their content understanding. High-quality professional development 
ensures both depth of content knowledge and a strong foundation in the pedagogy of the particular 
discipline. 

Instructional strategies. The professional development most likely to encourage teachers to 
examine and modify their practice with students is grounded in a "big picture" perspective on the 
purposes and practices of schooling (Little, 1993). Among the content features of effective 
professional development are the following characteristics (Darling-Hammond & McLaughlin, 1995): 

• It engages teachers in concrete tasks of teaching, assessment, observation, and 
reflection that illuminate the process of learning and development 

• It is connected to and derived from teachers’ work with their students 



Professional development on instructional strategies often target specific practices, such as inquiry- 
based lessons, use of manipulatives, cooperative learning strategies, and integrating technology. 
Activities also help participants determine where some tried and true methods may still make a 
contribution to student learning and where familiar routines can be adapted to new scenarios. 
Increasing skill in assessment related to new approaches is an important feature of high-quality 
professional development. Professional development may also facilitate access to and awareness of 
resources, both new and existing. 

Interpersonal/organizational skills. In its recent KEYS initiative, designed to help schools 
assess their quality for the purpose of developing reform strategies, the National Education 
Association found that some kinds of professional development were associated with effectiveness by 
virtue of improving organizational climate or efficacy. Members reported that learning process skills, 
such as brainstorming, assertiveness, consensus building, decision making, and problem solving 
helped them support the work of the school community. 



What are Effective Contexts for Professional Development? 

The process of restructuring schools places demands on the whole organization that make it 
imperative that both individuals and schools redefine their work in relation to each other (Lieberman, 
1995; Sparks, 1995). High-quality professional development jointly addresses the development of 
organizations and individual teachers within the context of the school’s culture and institutionalized 
supports. Successful professional development depends upon the willingness and capacity of all 
teachers and the school itself to establish and work toward goals for high student performance. When 
well-designed professional development programs succeed, schools become learning organizations in 
which people work together to design goals, solve problems, and create collective solutions. 

Institutional support. In a supportive environment, professional development is seen as an 
essential and indispensable tool for school improvement, rather than a frill that can be cut during 
difficult financial times (Sparks, 1995). Advocates argue that the public must support professional 
development as part of teachers’ regular work; the professional development standards of the National 
Council for Staff Development, for example, call for school districts to designate professional 
development as a line item in their school budgets (National Staff Development Council, 1994; 1995a; 
1995b). Institutional support also includes incentives for participation. These may be professional, 
for example increased opportunities for leadership or improved supervisory feedback, or material, for 
example stipends or salary increases. In order for these incentives to have their intended effects, it is 
important that they do not conflict with each other or set up competing expectations for teachers. For 



example, a district may provide stipends and instructional leadership roles to teachers who work on 
developing new district curriculum standards, while state recertification requirements specify that 
teachers must earn a certain number of college credits every five years to keep their certification 
current. Teachers must then decide how to allocate their limited free time to meet the competing 
demands of the state and the district. 

Financial support. At a practical level, at least modest financial support for professional 
development is necessary to ensure an adequate program of activities. Efforts to build and expand 
community backing for ongoing work and to tap external resources as needed highlight the value of 
professional development to overall school quality. 

School culture. School culture can have a strong impact on the extent and the quality of 
teachers’ opportunity to learn. In schools that work effectively as learning organizations, teacher 
learning is a collaborative enterprise, in which knowledge is shared and the focus is on teachers’ 
communities of practice rather than on individual teachers. Schools characterized by norms of 
collegiality and experimentation are more likely to succeed in helping students achieve at high levels 
(Lieberman, 1995). Teachers are more likely to persist in using new behaviors in school where 
collaboration and professional risk taking (and its occasional failures) are encouraged. 

Successful schools are organized as places for teachers to learn as well as to teach. Principals 
in these schools actively encourage teachers to invest in their own learning, and find the means for 
them to do so. Teachers have ample opportunity to collaborate on their work, and are encouraged- 
by the structure of the school day and the consistent support of their principal-to work together on 
problems of common interest. 

In such an environment, schools and districts ensure that teachers have the time and 
opportunity to get away physically and mentally from their daily work in the classroom. For the 
entire school and for individual teachers, reforming the educational process often extends over a 
lengthy period of time. For most schools, this will require not merely months but years. Moreover, 
much of what teachers need to learn requires sustained time, not an afternoon here and a Saturday 
mo rnin g there. For many kinds of professional development activities, this may require restructuring 
the school day or week to free up prime time for teachers’ learning and providing stipends to support 
teachers’ participation in evening and weekend activities. 
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What Outcomes Should Professional Development Seek? 



Effective professional development produces changes on a variety of levels: in the 
organization and culture of schools; in teachers’ knowledge, beliefs, and skills; in teachers’ practice in 
the classrooms; and, ultimately, in student achievement. Effective professional development enables 
teachers to work as professionals, able to apply broad principles of "best" practice to specific contexts 
and heterogeneous groups of learners. 

Teacher knowledge and attitudes. High-quality professional development enhances teachers’ 
knowledge of subject matter and pedagogy and increases their belief that all students can achieve to 
high standards, at least in part as a result of teachers’ own improved efforts. These are the explicit 
goals of national professional development activities in mathematics and science, most notably. High- 
quality professional development also gives teachers a deeper understanding of their students and how 
they learn. Teachers become better able to recognize and respond to students’ cultural diversity and 
differences in learning styles. Finally, teachers demonstrate high expectations for all students. 

Teacher classroom practice. High-quality professional development is designed to improve 
curriculum, instruction, and assessment practices in classrooms. As teachers attempt to engage 
students actively in authentic learning, they may use a redesigned curriculum with new teaching 
strategies— such as cooperative learning, hands-on activities, extended investigations, and new 
technology— that involve students in constructing their own understanding of a problem. A student- 
centered approach to teaching may use more time to cover less, necessitating choices about essential 
content. Teachers must also develop or adopt new assessment methods that probe students’ 
understanding of content and examine their ability to integrate knowledge and apply it to real life 
problems. 

School-level change. High-quality professional development encourages collaboration among 
teachers as they experiment with and reflect on their classroom practice. Schools become more 
collegial; school culture encourages teachers to develop new approaches to identifying and solving 
problems. Teachers work with each other to understand the effects of various curricula, instruction, 
and assessment practices on the desired student outcomes and to develop new practices that will 
enable their particular students to succeed. Through this process, schools come closer to achieving 
their improvement goals. 

Student achievement. Increased student achievement is the ultimate objective of professional 
development. High-quality professional development helps teachers engage students in learning. 
Students who are deeply engaged in well-conceived academic tasks are on the way to achieving 



meaningful educational goals. And as students work harder on focused instructional activities, their 
performance on measures of achievement aligned with the school’s vision of teaching and learning 



increases. 



III. Professional Development Indicators 



Adding professional development to the National Goals, emphasizing explicitly the importance 
of professional development in such federal programs as Title I, and launching a new recognition 
program for outstanding professional development programs (ED, May 1996) may signal the shift of 
policy support to this dimension of educational systems. Current conceptions of reform make 
professional development an integral component of improvements in the educational process. 

Effective and efficient professional development is essential in a context of high stakes and limited 
funding. In such an atmosphere, members of the education community— policymakers, parents, 
adminis trators, professional development providers, and teachers— need indicators with which to 
characterize professional development and its results. Such indicators are typically used to focus 
policymakers, educators, and the public on (1) key aspects of how an educational system is operating, 
(2) whether progress is being made, and (3) where there are problems (Blank, 1993). Unlike routine 
descriptive data, indicators contain a standard against which programs may be judged (Frechtling et 
al . , 1995). 

This set of indicators uses professional development design, delivery, content, context, and 
outcomes as the framework for assessing the relevance and availability of data collection instruments 
that focus on professional development programs. The construction of this taxonomy of professional 
development indicators builds on earlier work by Horizon Research, Inc., the ED Professional 
Development Team, Frechtling et al., National Staff Development Council Standards, and other state 
and local evaluations of professional development programs, although no existing set of indicators 
accommodates the broad range of needed perspectives on teaching quality related to reform. The 
following outline covers the dimensions that are key contributors to quality: 



I. Design 

A. Connection with broader school improvement goals 

• Links to a coherent plan, with clear and shared goals, based on school needs 

• Draws on a vision of teaching and learning that is congruent with appropriate 
standards 

• Integrates with other improvement efforts operating at the school 



B. Design elements 



• Focuses on organizational as well as individual improvement 

• Accommodates the need for understanding and skills to be built over time, 
reinforced continuously, and practiced in a variety of situations 

• Promotes continuous inquiry and improvement embedded in the daily life of 
schools 

• Addresses teachers’ evolving needs 

• Reflects input of those who will participate in professional development 
activities 

• Includes an evaluation process that is ongoing, focuses on relevant levels of 
the org anizati on, and informs future professional development activity 

• Ens ures equal access by teachers of disadvantaged or minority students 

Delivery 

A. Offers a range of roles and responsibilities with multiple ways to participate 

B. Provides for specific types of professional development activity 

• Training 

• Individually guided development 

• Collaboration 

• School improvement and curriculum development 

• Inquiry 

C. Provides activities of sufficient duration and intensity to meet goals 

D. Integrates professional community resources effectively 

Content 

A. Subject matter 



Presents imp ortant concepts and processes of disciplines accurately 



• Presents content aligned with relevant national, state, and local standards 

• Builds on teachers’ current knowledge of content, teaching, and lear nin g 

B. Instructional strategies 

• Models effective pedagogy, including appropriate use of inquiry-based 
instructional strategies; hands-on/manipulative materials; cooperative groups; 
technology; and direct instruction (review, lecture, demonstration, guided 
practice, independent practice) 

• Provides training in effective subject-specific pedagogy, i.e. instructional 
methods especially suited to conveying the content of a specific discipline 

• Provides effective strategies for recognizing and responding to student 
diversity, including involvement of parents 

• Provides effective assessment strategies, including performance assessment 

• Provides generalizable and practical resources, strategies, and techniques 

C. Interpersonal/organizational skills 

• Addresses skills required to function effectively under new organizational 
arrangements (e.g., shared decisionmaking, instructional teams) 



IV. Context 

A. Institutional support 

• Schools and districts provide adequate time during the school day for teachers 
to work together to accomplish the school’s goals 

• Schools and districts provide incentives to participate in professional 
development, e.g., stipends, salary increases, leadership responsibility, 
enhanced status, supervisory support 

• School, district, and state incentives to participate in professional development 
are designed to function coherently, so that incentives do not conflict with 
each other or set up competing sets of expectations for teachers 

• Schools and districts make it possible for teachers to attend to their 
professional development during the school day, e.g., by providing 
substitutes, rearranging the school schedule 



B. Financing 

• Builds community support for adequate funding 

• Develops external sources of support as needed 

C. School Culture 

• Supports teacher collaboration, interaction 

• Encourages teacher participation in designing professional development 
activities and in assessing need 

• Allows teachers to take responsibility for their own professional development, 
balancing individual and collective needs 



Outcomes 

A. Teachers’ knowledge/attitudes 

• Teachers show improved mastery of subject matter 

• Teachers express understanding of cultural diversity 

• Teachers demonstrate high expectations for all students 

B. Teachers’ classroom practice 

• Lesson content is aligned with appropriate discipline standards and matched to 
students’ developmental level 

• Teachers act as facilitators, maintaining a culture of inquiry 

• Teachers employ a repertoire of methods that engage students deeply in 
appropriate academic tasks 

• Teachers recognize and respond to student diversity, including involvement of 
parents 

• Teachers use assessments appropriate to the full range of learning goals, 
including those that support inquiry-based instruction, and use the results to 
improve instruction 
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School changes 



• Teacher collaboration increases 

• Schools meet their reform goals 
Student changes 

• Students perform better on measures of achievement aligned with the school 
community’s vision of teaching and learning 

• Disadvantaged students or students in schools with high concentrations of 
poverty improve performance 

• Students become more actively engaged in learning 



IV. Measuring Professional Development Process and Outcomes 



Attempts to document professional development activity have used a variety of methods. 
Likewise, a variety of means have been employed to demonstrate links between professional 
development activity and outcomes for schools, classrooms, teachers, and students. Data collection 
strategies have included teacher surveys, interviews, observations of teachers in classrooms and 
professional development activities, and analysis of student achievement test data. This section 
summarizes some of the lessons learned from these measurement activities. 



Teacher Surveys 

Properly designed, employed, and timed, teacher surveys are a relatively inexpensive and 
efficient way to gather descriptive data on many elements of professional development. Surveys of 
teacher participants can collect data on the design of professional development, especially its 
connection with broader school improvement goals. Teachers may be in the best position to judge the 
congruence of professional development with the school plan and with other school-based reform 
efforts. 



Surveys are an efficient means of gathering data on delivery and rates of participation in 
professional development. If results can be disaggregated, surveys will also yield information about 
the participation of teachers of special populations, such as disadvantaged or minority students. 
Surveys are a useful way to measure the quantity of professional development teachers receive— how 
many hours they spent in various kinds of activities, how recently they participated, and how long the 
activities extend over time. 

Surveys are also useful in collecting data on the content of professional development 
activities. Surveys can solicit information on the academic subjects addressed in professional 
development programs, as well as content-specific or general pedagogical practices (for example, 
teaching mathematical concepts using manipulative materials or cooperative learning). 

Teacher surveys can also gather data on the context within which professional development 
occurs, especially the institutional supports for professional development activities. Teachers can 
report on the policies in place to support their participation in professional development activities- 



stipends, salary increases, or paid release time. Surveys can also collect data on teachers’ opinions 
on school culture and how it supports or impedes professional development. 

Properly administered, surveys can also measure some elements of outcomes related to 
professional development. For example, surveys given before and after specific training can measure 
respondents’ perceptions of changes in practice, attitudes, and student outcomes. As with any data 
collection, however, the timing of both surveys is critical, since the pre-survey needs to be as close as 
possible to the beginning of the change period, and the post-survey given at an appropriate interval 
after the change could be reasonably expected to occur. Improper timing can lead to less than valid 
results. This is especially true when attempting to gauge the effect of a particular professional 
development event on classroom practice. Evaluating the extent of changes in teacher behavior and 
student outcomes is difficult and cannot be done very soon after a professional development 
experience, because many new practices take time to be implemented. However, surveys about some 
types of outcomes may be good sources of information about trends in perception and attitudes. 

Items used to measure teacher satisfaction at the conclusion of a training session yield less 
useful information because they tend to generate improbably high ratings from participants. It is 
difficult to interpret what teachers mean when they say an event was "good" or "useful.” It is 
sometimes apparent that high ratings reflect approval rather than acquisition of the desired knowledge, 
skills, or understanding. An engaging or amusing presentation or activity may elicit high approval 
ratings without necessarily producing the desired changes. 

National surveys are most useful in gathering information to inform broad questions such as 
might be developed to track change in professional development over time. However, as self-reports 
they are subject to types of response bias that limit accuracy and reliability. Framing low-inference 
items may reduce the effect of response bias, but may also reduce coherence and validity. Interviews 
offer a richer data collection strategy, but one that is also more labor intensive and therefore less 
practical to use on a large scale. 



Teacher Interviews 

Structured or semi-structured interviews with teachers allow for a different type of assessment 
of teachers’ experience with professional development and of changes in their knowledge and 
practices. Probing questions allow researchers to gather detailed data on the influence of professional 
development activities on teachers’ belief systems and-to a lesser extent-practices. This kind of 
inquiry provides a basis for understanding the effectiveness of specific types of learning opportunities 



and how the delivery of those opportunities benefits teachers most. One interview study of changes in 
teachers’ knowledge, for example, presented hypothetical teaching situations to teachers and asked 
them how they would respond to these. Because the situations were standardized, the interview 
format reduced the amount of irrelevant, idiosyncratic differences in responses that make comparisons 
difficult among subjects. However, because it allowed for open-ended responses, the format 
generated detailed, contextualized information about teachers’ perceptions of practice. In this case, 
the interview couched virtually all questions in the context of teaching situations, and thus it offered 
an opportunity to see how the various aspects of expertise— knowledge, beliefs, attitudes— about 
teaching, learning, and subject matter were drawn on to make teaching decisions (Kennedy, Ball, & 
McDairmid, 1993). 

Some studies of professional development programs conduct a series of interviews with a 
group of teachers over time. Interviews at regular intervals over the lifetime of a professional 
development project allow researchers the opportunity to question directly changes in teachers’ beliefs 
and knowledge of teaching and their students. Such interviews could reveal the extent to which 
opportunities for professional development emerge independently or as interlocking parts of a 
coordinated school program. Furthermore, inasmuch as professional development schools and other 
forms of coordinated, multiagency programs are increasing in number, this could be an important 
movement to track. For example, in Austin, Texas, professional development schools routinely 
include the faculty, teacher candidates, and graduate students from the local colleges of education and 
Americorps participants who are enrolled in a variety of degree programs but who work 15-20 hours 
a week in a single school for their entire undergraduate career. In Austin and other cities, many 
agencies participate in integrated service delivery programs that also play a role in professional 
development. Using survey data to first document emerging trends, case studies could then provide 
an important source of information for policy making. 



Observations of Teacher Practice 

Some evaluations of professional development observe how teachers’ curriculum, instruction, 
and assessment practices have changed, either by sending observers into classrooms or by analyzing 
videotapes of a specific lesson. Direct observations of teaching allow researchers to assess teachers’ 
classroom practices without relying on teachers’ perceptions of how their curriculum and instruction 
have changed as a result of their professional development. 

This method is not without liability, however. Among observers there may be a great deal of 
variability in perceptions of events. Researchers can increase the reliability of classroom observations 



by using tightly focused protocols or detailed checklists asking for evidence of discrete skills or 
behaviors. These kinds of observations may tend to oversimplify the tasks of teaching and may 
emphasize relatively trivial, low-inference tallies of behavior (Darling-Hammond, 1995). On the 
other hand, protocols that require observers to make judgments about what they see are very difficult 
to use reliably. For example, a protocol for an evaluation of the National Science Foundation’s Local 
Systemic Change Through Teacher Enhancement projects asked classroom observers to rate 
classrooms on whether "instructional strategies were appropriate for accomplishing the purposes of 
the lesson,” or whether "the design of the lesson encouraged a collaborative approach to learning.” 
The developers of this evaluation reported that they had no success in developing inter-rater reliability 
among observers using this instrument (Iris Wiess, personal communication, 2/12/96). 

Nevertheless, between the extremes of overly detailed, trivial checklists and overly general 
calls for judgment are studies that combine observations with other data collection strategies to 
enhance the validity of portrayals of practice while preserving reliability with cost-effective strategies 
used on larger samples (e.g.. Porter et al., 1995). Some studies use complex analytic strategies to 
construct reasonably coherent models of classroom activities from data collected with relatively low- 
inference items. In addition, some studies have used artifact collection as a proxy for observations, 
developing more accurate portrayals by "observing" the materials that shape the lessons of some 
teachers sampled from a larger group from whom survey responses were collected (Burstein et al., 
1995). The Third International Mathematics and Science Study (TIMSS, in progress) is testing an 
ambitious version of this combined approach, using curriculum analysis, teacher surveys, student 
surveys and testing, and videotapes of instruction to generate a comprehensive view of practice. 

These strategies hold promise for enabling researchers to extract more meaning from data bases 
intended to inform policy development. 



Observations of Professional Development Activity 

Some evaluations of professional development programs include observations of the 
professional development activity itself (e.g.. Firestone & Pennell, 1995; Horizon Research, Inc., 
1996). These observations allow researchers to develop detailed descriptions of the design, content, 
and pedagogy of specific professional development activities. Where other methods of data collection 
treat the professional development activity itself as a "black box,” or generic treatment given to 
teachers, observation of professional development allows researchers to form hypotheses about what 
kinds of learning opportunities are most likely to engage teachers and result in changed classroom 
practice. Observers can evaluate how teachers interact with each other and engage in the task at 
hand, the fit between the professional development strategies used and the goals of the session, and 



how well the session succeeded in engaging teachers in grappling with important ideas about teaching 
and learning. 

Such observations are most useful in evaluating specific professional development strategies or 
projects. However, if researchers need to understand the effects of all of the professional 
development activities available to teachers at a school, for example, they would be hard-pressed to 
make systematic observations of a range of widely varying activities. And, as in the case of teacher 
observations, ensuring reliability among observers is very difficult. 

Another major disadvantage of observations and interviews, of course, is the cost. 
Observations and interviews require a substantial investment prohibitive to most data collection 
efforts. For this reason, observations and interviews tend to be used in small studies or to validate 
information selectively from a larger data collection effort. 



Analysis of Student Achievement Test Results 

The ultimate goal of professional development is increased student learning. Policymakers 
and the public seek evidence that professional development is "working "-that students are learning 
more and performing better on publicly recognized measures of achievement as a result of the 
resources invested in teachers’ professional development. The most obvious and publicly credible 
way to look for improvement in student learning is to examine changes in student achievement test 
results. A few states participating in the National Science Foundation’s Statewide Systemic Initiative 
(SSI) have designed evaluations of their professional development efforts that incorporate student 
achievement data as evidence. The Kentucky SSI, for example, is tracking changes in student 
performance on state assessments in schools with SSI-trained and non-SSI-trained teachers. Greater 
increases in scores in SSI schools would provide convincing evidence that SSI training is making a 
difference. 

Finding evidence of increases in student achievement provides a strong argument for the 
efficacy of professional development, but it is nonetheless a difficult argument to make 
unambiguously. Among the problems are (1) the mismatch between most achievement tests and the 
visions of authentic, inquiry-based learning promoted by many professional development programs; 
(2) the challenge of establishing a direct link between professional development activities, 
improvements in teaching quality, and a rise in achievement test scores; and (3) the likelihood that 
several concurrent ”treatments"-professional development, new materials, and new organizational 
arrangements, for example— are leading to desired outcomes. 



Aligned Assessments 



Using student achievement tests to assess the effectiveness of professional development only 
works if the achievement tests are well-aligned with the kinds of teaching and learning targeted by 
professional development efforts. Most achievement tests measure some, but not all, of the goals of 
education; whether some tests measure even the most important outcomes is debatable. Thus, gains 
in tested student achievement may constitute a very narrow outcome for estimating the effects of 
professional development on student learning. Alternative assessments of student learning- 
performance tasks, portfolios of student work, paper-and-pencil tests containing open-response items— 
may provide a more complete picture of student learning. However, the reliability and validity of 
many of these new assessments have yet to be established when they are used in large-scale 
evaluations of student or school performance. 



Causality 



We have suggested that teacher quality is a primary in-school influence on student learning. 
However, other school factors directly influence student achievement (Brophy & Good, 1986) as well 
as teacher quality. Variation in school climate, community support for student learning, and school- 
level supports, such as the availability of high-quality teaching materials, may all affect both teaching 
quality and student achievement. The confluence of these variables makes it difficult to ascribe 
changes in student achievement directly to any one factor. 

Because teaching quality is a close and direct mediator of student learning opportunities, 
however, it must be seen as one important contributor to student achievement. Professional 
development is designed explicitly to enhance teaching quality. Measuring professional development 
enables policy makers to assess the vitality of one important dimension of education quality. Surveys 
are often used for this purpose. 



I 



V. Professional Development as Measured Through Existing Surveys 



I 

Previous sections outlined the connections between professional development and teaching 
quality that promote student learning. Using a taxonomy of indicators, we suggested measures that, 
in totality, capture the range of variables related to current conceptions of high quality professional 
^ development. In this section, we discuss how professional development has been measured through 

surveys and display the results of our search for professional development survey items, using the 
taxonomy of professional development indicators. 

We culled items from educational surveys representing a range of national, state, and 
■ university sources; more than twenty surveys were appropriate for this purpose. We sorted items by 

type, then compared stems, response options, and formats. We found substantial duplication; for 
instance, eight surveys used similar items to probe subject-matter content and six surveys asked for 
teachers’ perceptions of school culture. Within these, we found many variants of common items. 

• For example, in two items having identical stems, one offered response options measuring the 

frequency of the teacher’s participation, while the other measured the recency of participation. There 
were also gaps in the collection, with no survey items measuring a few elements of our taxonomy. 

9 This section, organized according to the indicator list, describes and provides examples of 

professional development survey items. Appendix D, arranged in the same order, includes selected 
relevant items. A list of surveys examined is included as the first page of Appendix D. 



Items About the Design of Professional Development 

Design issues that affect the quality of professional development cluster in two groups. One 
group reflects connections with broader school improvement goals. Linking professional development 
to a coherent plan, basing it on a vision congruent with relevant standards, and integrating it with 
other school reform efforts strengthen the likelihood that new approaches will be adopted and 
implemented sensibly. The other group covers design elements such as balance of focus on individual 
and collective improvement, provision for reflection on practice, evaluation of effects, and equitable 
access. These elements contribute rigor to a plan. 
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Very few current items ask specifically about professional development design or about issues 
from which one could infer dimensions of design. One item in the Schools and Staffing Survey (and 
repeated in Reform Up Close) asks the extent of teacher influence ("none" to "a great deal") over: 

(44b) Determining the content of in-service programs 

An item on the National Science Foundation’s Local Systemic Change Teacher Questionnaire 
also touches on design, among other things: 



(8) Generally speaking, to what extent has each of the following been true of science-related professional 
development you have experienced in your district in the past year? [Response options: "Not at an" (1) to 
"To a great extent" (5) and "Don’t know" (6)] 

c. Iam involved in planning my science-related professional development, 

i. Teachers in the district are involved in planning for professional development, 

k. Professional development activities draw on expertise of teachers in the district. 

National Science Foundation ' s Local Systemic Change Teacher Questionnaire 



If the quality of the design of professional development is to be examined, then more items 
directly addressing the relevant aspects are needed. 



Items About Delivery of Professional Development 

The delivery of professional development influences the extent to which the program has the 
desired effect, and delivery is the focus of almost half of the survey items. These items ask about: 
the range of opportunities for participation; the amount of followup, coaching, and evaluation that 
occurs after an activity; the adequacy of the activity’s duration and intensity to meet program goals; 
the extent of emphasis on inquiry, reflection, and collaboration; and encouragement to join informal 
professional learning groups. Most of these present either inventories of professional development 
options, from which respondents select those that apply to their experience, or questions about 
specific types of professional development activity. 



Range of Opportunities for Participation 

A sizeable portion (15 percent) of all items had the inventory format. Response options on 
such items offered respondents the chance to describe one of three dimensions of experience: (1) 



participation : e.g., "check those offered” in a given period; (2) recency: when the last experience of 
each type had taken place; or (3) duration: how much time was spent on each activity. 



A typical example of a recency participation inventory question, taken from the 1995 
Standards-Based Professional Development Program survey, is presented below. This item asks about 
teacher participation in a combination of fairly traditional delivery modes, such as workshops, 
education conferences, or university courses, as well as highly specific activities reflecting a broader 
view of professional development, such as participation on a school-based management committee. 

(12) What professional development experiences did you have in 1994-95? (CIRCLE ALL THAT APPLY) 

a. I attended a state or national education conference 

b. I attended a workshop on curriculum/instruction that was sponsored by the district or my school 
(including sessions associated with the Compact) 

c . j attended a workshop on school organization and management that was sponsored by the district or my 
school 

d. I took a graduate course through a college or university 

e. I served on a school-based management committee 

f. I read an education journal on a regular basis 

g. I met with colleagues on a regular basis to review and analyze student work 

h. I met with colleagues to develop curriculum, lessons, or assessments 

i. I made a presentation at a conference or other professional meeting 

j. Other (SPECIFY) 

Standards-Based Professional Development Program 



Specific Types 

About 30 percent of all survey items focused on teacher participation in specific types of 
development activities, many of which included evolving approaches to professional development. 
We categorize them here with the terms used by Sparks (1995), which overlap with our delivery 
indicators. 



Training 

The "training" category includes such activities as workshops after school or on student 
early-release days in which a presenter (often someone external to the school or district) aims to 
increase awareness, knowledge, and development of new skills. This type of professional 
development may cut across content areas and grade levels. It may be offered in response to 
participant requests or selected as an option by participants who have a special interest. Although 
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normally it would be included in a menu of offerings because the decisionmaker perceived it as 
responsive to general interest, it is not necessarily connected to the "big picture" of a school plan. 

The following example of a training item is taken from the 1990-91 SASS; it asks teachers 
about workshops or in-service training, their relevance to their field, and teachers’ reasons for 
participation. 



24a. 


In addition to the college courses you have already 
reported, have you ever participated in any teacher 
workshops or in-service training which included 30 
hours or more of class attendance? 


[098] 


1 [] Yes-Continue with b 

2 [] No— Skip to item 25a 


b. 


Was this training relevant to your current MAIN 
teaching assignment field? 


[099] 


1 [] Yes 

2 [] No 


c. 


What was your MAJOR purpose for taking this 
training? 


[100] 


1 [] To fulfill initial certification requirements for 

current position 

2 [] To maintain and/or improve qualifications for 

current position (including meeting recertification 
requirements) 

3 [] To train to teach a different subject matter area 

4 [] To train to teach at a different grade level 

5 [] To train to teach handicapped students or students 

with learning disabilities 

6 [] To train to teach other special student populations 

(e.g., Native American, limited English 
proficiency, etc.) 

7 [] To acquire credentials in new nonteaching areas 

(e.g., administration, guidance counseling) 

8 [] Other - Describe - 



SASS, 1990-91 



A s imilar item also appeared on 1987-88 version of SASS. NCES broadened its data 
collection in the 1993-94 version, after consulting with the National Education Goals Panel and other 
members of the education community about the need for more national information about teachers’ 
professional growth. Rather than concentrate narrowly on in-service activities, NCES expanded the 
number of items on professional development in the 1993-94 survey to examine teachers’ evaluation 
of such activities and their reports on support provided for professional development. 



Individually Guided Development 

In this arena of professional development, teachers individually identify learning needs, set 
their own goals, and engage in the activities with which they will meet their goals. Typical activities 
may include reading professional publications, collaborating with colleagues, and experimenting with 
new instructional strategies. Much of this activity is very informal, involving incidental or 
serendipitous learning rather than a planned program. Indeed, individual interpretation of exactly 



what constitutes individually guided development may be a drawback to interpreting responses to 
items with that (low) level of specificity. Perhaps for that reason, many items on individually guided 
development ask simply for the frequency or recency of the teacher’s participation in individual 
activities that are not necessarily connected with a school or district goal. Examples of such items ask 
about professional reading and "keeping up-to-date in my field. 

7. How often do your read professional journals? 

Frequently Occasionally 

Seldom Never 

8. Which of the following professional journals do you currently read? 

Journal of Reading 

Reading Teacher 

Reading Research Quarterly 

Language Arts Other None 

9. How often do you apply what you read in professional journals to your classroom practice? 

Frequently Occasionally 

Seldom Never 



Kentucky Title I Reading Teacher Survey 



Collaboration 



Most teaching has been a solitary occupation, in which collaborative effort with another 
professional was not a regular element. Including collaboration as a form of professional 
development explicitly recognizes the growth benefits accruing to teacher collaborators. This can take 
a variety of forms: investigating and solving a problem as a team; coordinating instruction between 
subject areas such that each builds on the other; combining classes to take advantage of different 
instructional strengths; or forming grade level or vertical program teams to improve continuity. Items 
about collaboration usually focus on the amount or the type. 

(37) Since the be ginning of this school year, how much time per month have you spent meeting informally 
with other teachers on lesson planning, curriculum development, or other instructional matters? 

(Circle one) 



Less than 15 minutes 1 

15-29 minut es 2 

30-59 minut es 3 

1 hour or more, less than 5 hours 4 

5 hours or more, less than 10 hours 5 

10 hours or more 6 



Reform Up Close 



Including survey items that probe teacher experiences with observation and assessment 
recognizes that professional development gains are sometimes realized from feedback based on an 
observation of teaching by a peer or supervisor. This observation can take the form of peer coaching, 
peer observation, or administrative observation for teacher evaluation. Whatever the purpose, the key 
defining element is that the teacher receives assessment or feedback that can support critical self- 
assessment and analysis. Response categories on survey items on observation or assessment are 
typically framed to identify the frequency of such activities. Examples of items, such as this one 
from Reform Up Close, ask primarily about non-evaluative observations by colleagues. 



(38) Since the beginning of the school year, how many times has your teaching been observed for purposes other than 
formal evaluation? 



Never 1 

Once 2 

Twice 3 

3-4 times 4 

5-9 times 5 

10 or more times 6 



Reform Up Close 



School Improvement/Curriculum Development Process 



This type of professional development usually involves teachers in a group process with the 
goal of solving a problem, improving instruction, or developing curriculum. Benefits may accrue 
both to the larger organization (e.g., grade level implementation of specific instructional strategies or 
schoolwide adoption of redesigned curriculum) and to the individual (e.g., new understanding of 
content, change in attitude or belief structure). Large-scale reform efforts, such as those involving 
the entire school or district, would be probed by items included in this category, but interestingly, we 
could find very few such items. Most questions asked about teacher participation in school 
improvement or curriculum development efforts, and many of these were included in larger items 
described as "inventories." 



How many hours outside of class do you spend 
during a typical week on the following activities: 



Zero 



Less than 

1 1-3 4-8 9+ 



LSC or LSC subcommittees 
PPAC or PPAC subcommittees 
Curriculum committees 
Other school committees 
Special Events/ Celebrations 



0 

O 

o 

0 

o 



o 

o 

o 

o 

o 



o 

o 

o 

o 

o 



o 

o 

o 

o 

o 



o 

o 

o 

o 

0 



Classroom Effects of Reform 



Inquiry 



In the context of professional development, inquiry refers to a systematic learning process by 
which teachers, individually or collectively, formulate research questions (generally about classroom 
practice), gather and analyze data to answer their questions, and then use the results to inform their 
own practice. Loucks-Horsley, et al. (1989) explain the usefulness of inquiry as a form of 
professional development. They note that teachers are intelligent, inquiring individuals who can 
develop new understandings that build on their previous expertise as they search for answers and 
formulate solutions to problems they have personally identified. Action research is a form of inquiry 
in which teachers try a new approach to some aspect of practice— sometimes an approach they have 
developed themselves-and then study its effects, with a view to improving effectiveness. It can be a 
productive way to discover "what works” in local contexts, focusing on observable and measurable 
outcomes of events under some local control. We found no surveys that included items about the use 
of inquiry in professional development. 



Duration and Intensity 

Among aspects of delivery that are not covered in Sparks’ system but that appear in our 
indicator list is the intensity and duration of professional development. This issue is addressed 
directly in items that appear in other categories of items. For example, the item immediately above 
could be used to measure not only the presence of collaboration but also its extent. Items about the 
content of professional development often provide response options in terms of time spent. Relatively 
few items ask for judgments about the adequacy of time spent, but adequacy may be inferred from 
responses to other items. One item that does probe directly is found on a Horizon Research 
questionnaire: 



(8) Generally speaking, to what extent has each of the following been true of science-related professional 
development you have experienced in your district in the past year? [Response options: "Not at all" (1) to 
"To a great extent" (5) and "Don’t know" (6)] 

m. Professional development programs are sustained over time, with ample followup activities and 
experiences. 

Local Systemic Change Teacher Questionnaire 



Overall, most of the dimensions of interest regarding delivery of professional development are 
addressed directly or indirectly by existing survey items. 



O 

ERIC 
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Resources for Professional Development 



Post secondary faculties engaged in preservice and continuing teacher education have been 
increasingly involved in collaborative efforts to improve practice, with central office and school-based 
partners and sometimes consultants from the private sector. Advocates of establishing "professional 
development schools" cite the benefits all around— to novices, veterans, and institutions, as well as to 
students whose opportunities to learn may be enriched. Such collaborations may serve to maintain the 
vitality of all the partners’ enterprises. We found no items that ask specifically about professional 
development schools or other joint efforts that focus on school programs. However, because this 
appears to be a growing trend, developing the capacity to track it may be useful. 



Items About the Content of Professional Development 

To improve teaching quality in areas related directly to student learning opportunities, 
professional development must address subject matter and general pedagogy. Subject matter elements 
include an accurate presentation of important concepts and processes, content that is aligned with 
relevant standards, and links to teachers’ current knowledge base. Instructional elements may include 
reform-oriented strategies (e.g., inquiry-based instruction, use of manipulatives, cooperative learning), 
strategies for engaging student diversity, standard and alternative forms of student assessments, 
appropriate use of familiar methods, and finding resources for teaching. 



Subject Matter 

About 15 percent of the survey items focused on the actual content and concepts presented in 
professional development programs. Most of the items in this category are drawn from content- 
specific surveys, such as the NAEP instruments for mathematics teachers. They typically offer an 
inventory of topics, with response options for duration or frequency. From such responses, one could 
determine whether teachers are studying important concepts and whether the topics align with 
standards advanced by professional associations, such as the National Council of Teachers of 
Mathematics. This item from the National Assessment of Educational Progress teacher survey 
represents one common form. 



What level of exposure, if any, have you had to each of the following topics or areas? 



1. Methods of teaching 
elementary mathematics 

2. Number systems and 
numeration 

3. Measurement in 
mathematics 

4. Geometry 

5. Probability/ statistics 

6. Abstract/ linear algebra 

7. Calculus 



One or more college Part of a college or 
or university courses university course 



In-service 

training 




Little or no 
exposure 

O 

O 

O 

O 

O 

O 

O 



National Assessment of Educational Progress Math Teacher Survey 



Instructional Strategies 



Professional development that enhances teaching quality with respect to repertoire 
addresses the need to stimulate student engagement in academic work. Issues of interest include 
whether such professional development models effective pedagogy, promotes appreciation of diverse 
student resources and abilities, increases participants’ skill in assessing student learning, and identifies 
sources of material support. Items about subject matter instructional strategies were also drawn 
primarily from content-specific surveys. Few items ask about availability of the resources necessary 
for implementation. Like the items about delivery, items about strategies use an inventory approach. 
The NAEP teacher survey item below is typical. 



Questions 1-14. During the past five years, have you ever had training in any of the following, either in 
college courses or in in-service education? Fill in one oval on each line. 





Yes 


No 


Study strategies 


[A] 


[B] 


Motivational strategies 


[A] 


[B] 


Teaching critical thinking 

The role of students' prior knowledge in their 


[A) 


[B] 


reading 


[A] 


[B] 


Diagnosis and remediation of reading difficulties 


[A] 


[B] 


Ability grouping 


[A] 


[B] 


Literature-based reading instruction 


[A] 


[B] 


Reading assessment 


[A] 


[B] 


Content area reading 


[A] 


[B1 


Combining reading and writing 


[A] 


[B] 


The whole language approach to teaching reading 


[A] 


IB] 


Phonics in the teaching of reading 


[A] 


[B] 


Individual reading programs 
Teaching students from different cultural 


[A] 


[B] 


backgrounds 


[A] 


IB] 



National Assessment of Educational Progress Reading Teacher Survey 
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Interpersonal/Organizational Knowledge and Skills 



A recent survey of the National Education Association, developed for the KEYS project, 
included an item asking about skills that many think are essential for participating in shared 
governance and team teaching. 



(16) How often is the following training offered in your school? 







Monthly or 


Several 

Times 


Once 


Every Few 


Not 


a. 


Assertiveness training 


More Often 
1 


A Year 
2 


A Year 
3 


Years 

4 


Offered 

5 


c. 


Brainstorming techniques 


1 


2 


3 


4 


5 


e. 


Communication skills 


1 


2 


3 


4 


5 


a 

O' 


Conflict resolution 


1 


2 


3 


4 


5 


h. 


Consensus building 


1 


2 


3 


4 


5 


m. 


Decisionmaking skills 


1 


2 


3 


4 


5 


0. 


Leadership skills 


1 


2 


3 


4 


5 


s. 


Problem solving skills 


1 


2 


3 


4 


5 


u. 


Teamwork training 


1 


2 


3 


4 


5 



National Education Association 

Items About Context for Professional Development 

Realizing the goals of professional development is often as dependent on context support as it 
is on design and content. Adapting new approaches requires institutional, financial, and cultural 
backing. Items found in our surveys address two aspects of support. Institutional support takes the 
form of time "on the clock" for professional development and material and professional incentives for 
participation (e.g., as release time, stipends, salary increases, leadership responsibilities, improved 
supervisory feedback). Support in the school culture may nurture teacher collaboration, permit 
participation in designing professional development, and encourage teachers to take responsibility for 
their own professional development. A third important form of context support is funding. Reports 
of major professional development initiatives have cited the historic problem of funding: when 
resources fail to keep pace with needs, professional development is often among the first activities to 
be curtailed. However, no items about this aspect of context were found. 



Institutional Support 



About 14 percent of the survey items inquire about school policies and practices that support 
professional development programs. Examples include this one from the Upgrading Mathematics 
Teacher survey, which was replicated in many other surveys: 

(11) What type(s) of support have you received for participating in formal in-service programs? 

(Circle all that apply.) 



General Math 



None 1 1 

Released time from teaching 2 2 

Travel and/or per diem expenses 3 3 

Stipends 4 4 

Professional growth credits 5 5 

Other (please specify) 6 6 



Upgrading Mathematics Study 



Other survey items inquire about practices that support more informal types of professional 
development, such as having common preparation or planning time. 

Financing 

We found no it ems that collect data on how professional development activities are funded by 
schools, districts, communities, or other external sources of support. While this is an important 
element in understanding the overall picture of professional development, such data may be less 
appropriate for surveys, and more easily gathered through other means such as interviews. 



School Culture 



The most co mm on survey item on school culture typically seeks teacher perception on specific 
institutional aspects that affect overall atmosphere. Norms supporting collaboration are often targeted. 
Six surveys include such items, collecting data on up to 26 specific elements thought to affect school 
culture. For example, two SASS items ask about teacher behaviors that are essentially voluntary and 
that reveal professional norms: 



(T47) Do you agree or disagree with each of the following statements? 



f. 


Teachers participate in making most of 
the important educational decisions in this 


Strongly 

Agree 

□ 


Somewhat 

Agree 

□ 


Somewhat 

Disagree 

n 


Strongly 

Disagree 

□ 


p- 


school. 

There is a great deal of cooperative effort 
among the staff members. 


□ 


□ 


□ 


□ 



1993-94 Schools and Staffing Survey , Teacher Questionnaire 



Items About Outcomes 



The focus of interest in this area is whether professional development outcomes are observed 



Teachers’ knowledge and attitudes about content, cultural diversity, and expectations for all 
students 

Teachers’ classroom practice in which content is aligned with standards; teachers employ a 
repertoire of methods appropriate to the subject and students; and teachers use assessments 
that support the full range of learning goals 

School-level changes such as institutional reform goals 

Student-level changes such as improving performance on appropriate measures of 
achievement, narrowing the achievement gap between disadvantaged students and others, and 
engaging students more actively in learning 



Few surveys attempt to assess the effect of professional development through items asking 
whether teacher behavior changed as a result of their participation. Similarly, few have sought data 
on whether teachers apply their new learning in the classroom. Such questions are subject to 
response bias and misperceptions, which limi ts the usefulness of the data they generate. Therefore, 
the shortage of such items may reflect some experts’ concern about the validity of teacher reports of 
behavioral changes. One example is adapted from the CER: 



Please rate the extent to which 
the following have changed in 
the past three years 



My teaching effectiveness 

Curriculum quality 

Professional growth opportunities 

Sense of community in the school 

My commitment to the school 

The quality of student academic performance 

Student behavior 

The school’s relations with parents 
How students get along with other students 
How parents get along with teachers 
How the school relates to the community 
How teachers get along with students 



No 

Worse Change Better 



[] 

[] 

[] 

[] 

[] 

[] 

[] 

[] 

[] 

[] 

[] 

[] 



[] 

[] 

[] 

[] 

[] 

[] 

[] 

[] 

[] 

[] 

[] 

[] 



Classroom Effects of Reform 



Noticeably missing from this list of items measuring outcomes are items about "teacher 
satisfaction" with professional development. While personal satisfaction is most definitely an element 
of a productive and engaged workplace, knowledge of what participants report about satisfaction 
sheds little light on the presence or absence of high quality outcomes from professional development, 
not even outcomes related to improved teacher attitudes. Surveys of staff development from the 
1970’s focused almost exclusively on recipient satisfaction with in-service programs (Sparks, 1995), 
yet did not clarify the inherent ambiguities. The complexity of forms of professional development 
today and the number of dimensions on which any one might be deemed "satisfactory" make it 
inevitably difficult to interpret data on satisfaction. Because of this, recent studies do little to assess 
directly teacher attitudes toward professional development, choosing instead to assess less ambiguous 
indications of teacher support and approval. 



Summary of Existing Survey Items 



In general, some areas of current interest are already well covered by items whose use has 
been sufficiently extensive to produce insight about their data collection properties. These areas 
include the delivery of professional development, its curriculum and instructional content, and some 



aspects of institutional support and school culture. Design elements, financial support, and outcomes 
related to student achievement or the teaching qualities that contribute to achievement are less often 
the focus of survey research. 



VI. Recommendations for NCES Data Collection 
on Professional Development 



NCES could enhance its capacity to characterize and report on professional development as a 
component of educational systems nationwide by extending current efforts in key ways and by making 
relatively few additions to its existing data collection efforts. In this section, we propose 
measurement strategies for the five dimensions we have identified as important: design, delivery, 
content, context, and outcomes. We organize our recommendations using the four types of data 
collection strategies described in Section HI above (and repeated in Table 1 below): surveys, 
interviews, observations, and analysis of student performance. Within our discussion of surveys, we 
recommend modifications to existing SASS items, suggest additional SASS items to improve the range 
of data collected, then suggest supplemental surveys where appropriate. We follow a similar 
reporting process with the remaining data collection strategies. In developing revised or new items 
for this section, we borrowed heavily from the surveys listed in Appendix D and from a recent paper 
by Dorothy Gilford (1996). 

To make these recommendations most useful to NCES, we followed three guidelines. First, 
we strove to maintain continuity of NCES data collections to permit comparisons over time. Second, 
we pl anne d our recommendations with budget considerations in mind, since they would necessarily 
limit expansive modifications. Finally, when we contemplated extending or contracting the 
boundaries of collected data to inform policy making in particular directions, we considered both the 
emerging developments and the enduring issues of professional development. 

In Table 1 , we s ummari ze the data collection strategies we recommend for each dimension of 
the professional development taxonomy. We next describe the recommendations in greater detail. We 
conclude this Section with a summary of our specific recommendations for amending current items in 
SASS and creating new ones. 



Table 1: Data collection methods recommended for collecting information on the dimensions of 
professional development outlined in the taxonomy. 



Dimension of 

Professional 

Development 


Data Collection Methods 


Surveys 


Interviews 


Observations 


Student 

Performance 


SASS 


Supplemental 


Teacher 

Practice 


PD Activity 


Design. 


✓ 




✓ 




✓ 




Delivery 


✓ 


J 


✓ 




/ 




Content 


✓ 


J 






/ 




Context 


✓ 


J 


✓ 




/ 




Outcomes 


✓ 


J 


✓ 


✓ 







Measuring Design 



Few existing survey items or data collection methods address the design of professional 
development directly. This is unfortunate since the design provides the foundation for effectiveness, 
ensuring that the knowledge and skills cultivated through the professional development have a direct 
bearing on the needs of the organization, its members, and their ability to achieve their mission. 

Three aspects of design can be measured initially with surveys, then described more fully with 
related studies: connection with larger school programs, responsiveness to teachers’ needs, and the 
extent to which evidence of effectiveness is gathered. In order to produce a long-lasting effect that 
matters in the broader context of schooling, professional development should arise from a coherent 
plan that addresses school needs. To stimulate teachers’ learning and maintain their competence in 
educating students, professional development should generally accommodate the teacher-learners’ 
range of prior knowledge and potential use of new information or skills. Provision should be made 
for all instructional staff: beginning teachers; those who are ready for advanced opportunities; and 
those whose roles are tangential or complementary to the teacher-learners’ roles. Additionally, to 
ensure the quality of the professional development experience, some form of ongoing evaluation 
should determine whether the investment of resources is generating the desired results. Evaluation of 
professional development may well focus initially on whether it leads to improvement of teachers’ 
knowledge and skill. Although improvements in students’ knowledge and skills are the ultimate goals 
of professional development, an unambiguous connection with teachers’ professional development will 
be more difficult to document. 



Surveys 



SASS. Currently, SASS collects little information on the design of professional development 
activities. We recommend continuing to use Item T44b on the extent of teacher influence on design, 
and gathering additional data by making minimal changes to current items in the Teacher 
Questionnaire, and by adding new items to the Principal Questionnaire. 1 

Item 30 in the current teacher questionnaire should be modified to collect additional specific 
data on professional development design. Adding an additional option (in italics below) would 
identify possible links between professional development activities and broader school improvement 
goals (Taxonomy element LA). 



(T30) Since the end of last school year, in which of these activities related to teaching have you participated? Mark (X) to all 
that apply. 

1 n School District sponsored workshops or in-service programs 

2 [I] School Sponsored workshops or in service programs 

3 CD University extension or adult education courses 

4 n College courses in your subject field 

5 d] Professional growth activities sponsored by professional associations 

6 IZ] Committee to integrate academic skills into the vocational curriculum 
7 IZ] Other curriculum committee 

8 IZ] Committee on selecting textbooks or materials 

9 IZ] Professional growth activities directly related to implementing a school improvement plan 
0 IZ] None of the above 

SASS 93-94 

Adding a statement (modified from the Local Systemic Change Teacher Questionnaire) to item 
47 in the current Teacher Questionnaire would collect data on the extent to which the design of 
professional development accommodates the need for teacher understanding and skills to be built over 
time, reinforced continuously, and practiced in a variety of situations (Taxonomy element IB). 



2 In this section, we identify items according to their particular SASS questionnaire. Existing items in the 
Teacher Questionnaire, for example, are labeled T30 or T47, corresponding to items 30 and 47. Proposed new 
items are also identified with the particular survey for which they are recommended. TX1 and PX1, for example, 
signify proposed items for the Teacher and Principal Questionnaires, respectively. 
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(T47) Do you agree or disagree with each of the following statements? 





Strongly 


Somewhat 


Somewhat 


Strongly 


Professional development activities are sustained 


agree 


agree 


disagree 


disagree 


over time, with ample participant followup and 
teacher support. 


□ 


□ 


□ 


- 



SASS 93-94 



New items could also be added to the SASS Administrator Questionnaire to measure 
additional dimensions of design (Taxonomy element IB). The first item below identifies elements of 
the school’s process for evaluating the effectiveness and design of professional development activities. 
The second collects information on multiple elements of professional development design. 



(PX1) Please identify the strategies your school uses to gather information on the effectiveness of professional development: 
Mark (X) to ail that apply. 



lU Surveys asking for teachers’ perceptions of professional development program effectiveness 
2 □ Observations of changes in teachers’ practice 
sD Artifact collection illustrating implementation of new practices 
4 1Z] Narrative reports of related changes in teaching 

5 D Portfolios documenting new teacher behavior or knowledge 

6 CD Videotapes of new practices 



(PX2) Please indicate which of the following statements describe your school-sponsored professional development programs. 
Mark (X) to all that apply. 

Professional development programs... 

are planned to meet the needs of teachers at different career stages 
are sustained over time, with ample followup activities and experiences 
provide for and encourage continuous inquiry 
are tied to school improvement plans 
reflect significant teacher input 
have content aligned with broad district goals 
are evaluated formally for evidence of effectiveness 

Supplemental surveys. Survey data on the design of professional development activities can 
be collected at sufficient depth through SASS, and we recommend no additional survey data 
collection. 



iD 

iD 

3 □ 

4 □ 

5 □ 

eD 

tD 



Interviews 



Clear, detailed knowledge of the design could be gained from the right collection of 
interviews at a given site. Interviews with district and school personnel, and with professional 
development trainers could be conducted efficiently using semi-structured interview protocols in 
telephone interviews or site visits with key planners or implementers of programs and a few 
participants. A semi-structured protocol would permit data collection that captures some low- 
inference, comparable aspects of practice while preserving the opportunity for informants to 
communicate their conceptions of learning experiences for teachers. Sites could be chosen on the 
basis of the similarity of their survey response patterns to high-frequency patterns evident from 
analysis of the whole sample, as illustrative rather than random cases. Mandel (1995) and Gilford 
(1996) also recommend using case studies to help define SASS questions about the range and 
character of professional development. 



Observations 



Some of the essential elements of design could be garnered by analyzing observational data 
obtained during a particular professional development activity. Such data could provide information 
on how well the particular design accommodates the need for understanding and skills to be built over 
time, practiced in a variety of situations; reflects input of participants; and ensures equal access by 
teachers of disadvantaged or minority students. Little information about design would be obtained by 
observing teacher practice. 



Student Performance 

Collecting data on student performance would not inform our understanding of how 
professional development is designed. 



Measuring Delivery 

The delivery of professional development influences effectiveness through provision of an 
adequate range of entry points and roles; its inclusion over time of opportunities suited to a spectrum 
of school structures and purposes (e.g., workshops, networks, committee work, inquiry); the 



adequacy of its duration and intensity; and its involvement of various sectors of the education 
community such as university faculty, central office specialists, subject matter experts. 

Information on the delivery of professional development can be adequately gathered using 
large-scale surveys, although more intensive surveys of specialized groups of teachers might yield 
more indepth information relative to specific content areas. 



Surveys 



SASS. Currently, SASS includes an item (T30) that addresses two of these four elements of 
delivery— a range of roles and responsibilities with multiple ways to participate (Taxonomy element 
IIA) and the specific types of activities (Taxonomy element ECB). No current SASS items examine the 
adequacy of professional development activities (Taxonomy element IIC), or their integration of 
community resources (Taxonomy element HD). Two relatively simple modifications to item T30, 
however, will collect additional information for IIB and address IIC. Expanding the types of 
professional development activities queried in the item will respond to Taxonomy element IIB, while 
replacing the current single (yes/no) response option with time and adequacy response options will 
accommodate element DC. Using appropriate analytic methods with this modified item will generate 
answers to multiple queries of the most important delivery elements. New elements are in italics. 



(T30) Since the end of last school year, in which of these activities related to teaching have you participated, how much time 
have you spent on each, and was the amount of time adequate? 







0 Hours 


8 Hours 
or less 


9-32 Hours 


More than 
32 hours 


Adequate 


1. 


School District sponsored 
workshops or in-service 
programs 


□ 


□ 


□ 


□ 


□ Yes 


□ No 


2. 


School Sponsored workshops 
or in service programs 


□ 


□ 


□ 


□ 


□ Yes 


□ No 


3. 


University extension or adult 
education courses 


□ 


□ 


□ 




□ Yes 


□ No 


4. 


College courses in your 
subject field 


□ 


□ 


□ 


□ 


□ Yes 


□ No 


5. 


Professional growth activities 
sponsored by professional 
associations 


□ 


□ 


□ 


□ 


□ Yes 


□ No 


6. 


Committee to integrate 
academic skills into the 
vocational curriculum 


□ 


□ 


□ 


□ 


□ Yes 


□ No 


7. 


Other curriculum committee 


□ 


□ 


□ 


□ 


□ Yes 


□ No 


8. 


Committee on selecting 
textbooks or materials 


□ 


□ 


□ 


□ 


□ Yes 


□ No 


9. 


Professional growth activities 
directly related to 
implementing a school 
improvement plan 


□ 


□ 


□ 


□ 


□ Yes 


□ No 


10. 


Single event workshops with 
little or no followup 


□ 


□ 


□ 


□ 


□ Yes 


□ No 


11. 


Individual activities you 
planned 


□ 


□ 


□ 


□ 


□ Yes 


□ No 


12. 


Collaboration with other 
teacher(s) at your school 


□ 


□ 


□ 


□ 


□ Yes 


□ No 


13. 


Individual inquiries in which 
you identified and researched 
a content or instructional 
topic of interest 


□ 


□ 


□ 


□ 


□ Yes 


□ No 


14. 

15. 


Other 

None of the above □ 


□ 


□ 


□ 


□ 


□ Yes 


□ No 



The extent to which community resources are used in professional development (Taxonomy 
element HD) could also be assessed through a new SASS teacher item. A modified item from the 
Scholastic Research Institute Pilot Teacher Survey may be appropriate: 



(TX1) ’On a scale of 1 to 5. to what extent did the professional development activities you participated in since the end of last 
school year involve the following local community resources? 



Not at 
All 

a. Other teachers 1 

b. Subject specialists or experts 1 

c. Business/industry professionals 1 

d. College/university professors 1 

e. Other college/university educators 1 

f. Community representatives 1 



A Great 
Extent 

2 3 

2 3 

2 3 

2 3 

2 3 

2 3 
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Supplemental surveys. Delivery of professional development can be measured by a 
supplementary survey of teachers within specific high-interest content areas asking for more details 
than could be gathered with an item that is sufficiently general for inclusion in SASS. For example, 
in targeting teachers in high-profile subjects in alternating years (such as mathematics one year and 
science the next year), responses to content-specific survey items might reveal important patterns in 
the use of external resource people: What experts do schools tap for various kinds of subject-matter 
workshops? Is there any change over time in the extent of use of university consultants, private 
contractors, and community partners? The delivery possibilities are sufficiently well known that it 
should be possible to construct a short content-specific survey in a multiple-choice format. Such a 
survey could enhance our view of the offerings, their delivery and change over time without 
sacrificing quality or generating too great a response burden. 



Interviews and Observations 



Combining interviews and observations into long term case studies tracking professional 
development over a year or longer at several sites chosen for representativeness could shed light on 
how configurations of opportunities develop and are sustained (or not). This information might be 
useful in understanding the interplay of forces within a system attempting broad reforms. Intensive 
case studies could be used to characterize professional development over a span of time using data 
collected in several ways over a short period. Observations of teacher practice would not materially 
aid our understanding of professional development delivery. 



Student Performance 



Collecting data on student performance would not inform our understanding of how 
professional development is delivered. 



Measuring Content 

Whether professional development activities influence educational outcomes depends in part 
on their content, and we have suggested that three types of content are most relevant to contemporary 
school improvement efforts: subject matter (e.g., mathematics, history); instructional strategies, 
including assessment; and process skills essential to functioning effectively in shared decisionmaking. 



Surveys 



SASS. With modification, one of the current SASS teacher items on professional 
development could provide information on all three aspects of content (Taxonomy dimension ID), plus 
d a ta on teachers’ perceptions of the outcomes (Taxonomy dimension V). Currently, item T31 
supplies some information on content areas (Taxonomy element IDA), instructional methodologies 
(Taxonomy element MB), and duration (Taxonomy element ICC), while item T32 provides outcome 
Hata (Taxonomy element VA and VB) generalized across all professional development programs. As 
Gilford (1996) suggests, combining T31 with T32 could create the potential to collect additional 
content data and more specific information on respondents’ opinions about the results of specific types 
of professional development. Such a merged item could be the following: 

(T31/T32) For professional development programs focusing on each of the topics listed, indicate which you have participated in 
since the end of last school year by writing first the amount of time spent (in total hours) and second whether outcomes listed 
were achieved. 



Time spent in This pro fessional development activity . . . 

activities since the (Indicate yes or no for each outcome) 



Focus of Content 


end of last year 
(to the nearest 
hour) 


...provided 
information that 
was new to me. 


...changed my 
views on 
teaching. 


...caused me to 
change my 
teaching 
practices. 


...caused me to 
seek more 
information or 
training. 


...were generally 
a waste of my 
time. 


All professional 
development activities 


hits) 


3 Yes 


G No 


□ Yes 


□ No 


G Yes 


□ No 


□ Yes 


□ No 


□ Yes 


□ No 


Uses of educational 
technology for 
instruction (e.g., use of 
computer, satellite 
learning) 


hi<s) 


3 Yes 


□ No 


□ Yes 


□ No 


□ Yes 


□ No 


□ Yes 


□ No 


□ Yes 


□ No 


Methods of teaching 
your subject field 


hr{s) 


□ Yes 


□ No 


□ Yes 


□ No 


□ Yes 


□ No 


□ Yes 


□ No 


G Yes 


G No 


In-depth study in your 
subject field 


hi<s) 


□ Yes 


□ No 


□ Yes 


□ No 


□ Yes 


□ No 


□ Yes 


□ No 


□ Yes 


□ No 


Student assessment (e.g, 
methods of testing, 
evaluation, performance 
assessment) 


hr{s) 


□ Yes 


□ No 


□ Yes 


□ No 


□ Yes 


□ No 


□ Yes 


□ No 


□ Yes 


G No 


Cooperative learning in 
the classroom 


hi<s) 


G Yes 


□ No 


□ Yes 


□ No 


□ Yes 


□ No 


G Yes 


□ No 


□ Yes 


□ No 


Methods of teaching 
students with limited 
English proficiency 


hi(s) 


>3 Yes 


G No 


□ Yes 


□ No 


□ Yes 


□ No 


□ Yes 


G No 


□ Yes 


□ No 


Classroom management 
skills 


hi(s) 


□ Yes 


□ No 


□ Yes 


□ No 


□ Yes 


□ No 


□ Yes 


G No 


□ Yes 


□ No 
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Including element a. All professional development activities continues the time series for 
current item T32; including element g. Methods of teaching students with limited English proficiency 
allows us to eliminate current item T63. 



In addition, one new item might offer insights about whether professional development builds 
on teachers’ current knowledge of content, teaching, and learning (Taxonomy element IIIA). 



(TX2) Please indicate the extent to which you agree that the following statements are true of your professional development 
opportunities during the last school year. 



Professional development programs 
recognize and build upon individual 
teachers' knowledge about topics as 


Not at 
A 11 








Completely 


they enter the program. 

Professional development programs 
assume that all teachers have the 


l 


2 


3 


4 


5 


same level of knowledge. 


1 


2 


3 


4 


5 


Professional development programs 


promote collaboration among 
teachers. 


1 


2 


3 


4 


5 



Supplemental surveys. Surveys could complete our understanding of the specific content of 
professional development and its potential to improve education workforce competence in key 
subjects. For example, math and science are areas where new content is emerging daily and 
significant numbers of faculty are teaching out of field, generating an obvious need for even teachers 
who were originally well prepared in their fields to pursue further study. Because of these factors, 
continuing education in disciplinary content may play a key role in the quality of teaching. Surveys 
targeting subpopulations of teachers in different fields could produce more detailed portrayals of 
content knowledge development in each discipline than could items on a general survey. 



Interviews 



Interviews with district and school a dminis trators and teachers involved in pl annin g 
professional development could round out the whole picture of professional development content, 
answering such questions as: How are topics for professional development chosen? What forms of 
professional development spin off from what forms of reform activity? However, the details of 



Observations 



One of the primary goals of professional development is to change teachers’ knowledge and 
attitudes and their classroom practices. Observational data can provide important primary data here. 
Observing teacher practice is likely to generate important information that could support or refute 
hypotheses about teacher learning and change in teacher behaviors attributable to professional 
development. 

Comb ining interviews and observations into case studies offers the lowest-inference strategy 
(in an inevitably high-inference game) to gather evidence to direct and support national policy 
decisions. Greater insight about actual practices could be gained and used to support policy 
development at all levels. 



Student Performance 

The ul timate goal of professional development is to improve student learning, and measuring 
student performance in relation to the professional development undertaken by teachers, and their 
changed instructional practices, offers direct evidence to make those theoretical connections explicit. 
Moreover, substantiated increases in student achievement unambiguously linked to specific 
professional development experiences and changed classroom behavior, offers a strong public 
statement about the value of professional development. 



practices (Taxonomy element VB), and, if modified according to earlier suggestions, it could provide 
even more comprehensive data. New items could assess these changes more completely, along with 
possible school-level changes. Suggestions for new items follow. 



(TX3) Check any of the changes you have observed in your own teaching that are the result of professional development you 

have participated in this past year: 

□ Use of new instructional strategies 

□ Use of new curriculum 

□ Use of new assessments 

□ Use of new instructional materials. 

The 1993-94 SASS also measures elements of school change (Taxonomy element VC) in item T30 
(teacher collaboration), but has no provision to measure student changes (Taxonomy element VD). In 
fact, linking professional development and changes in student achievement outcomes necessarily goes 
beyond the current conception of SASS. As explained in Section II, measuring this taxonomy element 
can only be accomplished by combining measures of student learning with measures of teacher 
development and instructional practices. 

Supplemental surveys. Using a survey with more in-depth items asking for more details from 
a targeted sample of teachers would greatly increase the specificity of data that could be collected. 
Targeting high interest content areas, such as mathematics and science, could significantly increase 
our unders tandin g of the specific ways in which teachers’ knowledge and attitudes are improved, their 
classroom actions changed, and the school culture improved. Many, but not all, of these changes 
may be content-specific; specialized surveys can collect that type of information. 



Interviews 



Telephone interviews based on semi-structured protocols with a diverse sample of teachers 
could result in a usefully rich description of how teachers change as a result of professional 
development. While the data are still subject to the unreliability introduced by response bias and 
faulty memory, some of the ambiguities can be reduced by asking the right questions in the right 
order and by probing for clarity. 



Supplemental surveys. A supplemental and more specialized survey would be helpful to track 
this dimension, particularly some of the important aspects of financing. Much of that data collection, 
such as that proposed by Chambers (1996), is conducive to supplemental surveys. 



Interviews and Observations 



The interplay of institutional support, financial support, and school culture is not well 
documented, yet it appears to be a key ingredient of systemic reform. Selected case studies of sites 
that fit common profiles observed in survey responses could offer insight into how those three factors 
operate with respect to teaching quality. Blending teacher and principal interviews with school 
observations of a random subsample of teachers, their principals, and their schools could result in a 
richer data set and a better characterization of that interplay in a manageable number of sites and 
periods of time, than surveys alone could provide. 



Student Performance 



Collecting data on student performance would not inform our understanding of the context of 
professional development. 



Measuring Outcomes 

Educational outcomes related to practice or professional learning are very difficult to measure 
reliably with surveys, but it seems nevertheless important to try, because alternative measurement 
strategies are time-cons umin g, burdensome to respondents, and costly. The professional development 
outcomes most proximate to student learning concern teacher attitudes and practice, which underlie 
teaching quality. Linkages to student achievement outcomes will be hard to prove, but a case can be 
sketched with data from teachers’ reports of their perceptions of change. 



Surveys 



SASS. The 1993-94 SASS collects data on two of the four elements of outcomes. Currently, 
item 32 of the 1993-94 SASS Teacher Questionnaire provides some information on changes in teacher 
attitudes and knowledge (Taxonomy element VA), along with the possible impact on classroom 



(PX3) Do you agree or disagree with each of the following statements? 



a. Evaluations of the principal are based, in part, 
on the school’s professional development 
activities. 

b. Participation on a school-based committee that 
works on curriculum, instruction, or assessment 
earns a teacher benefits typically associated with 
traditional professional development. 

c. Teacher evaluations are tied to their professional 
development activities. 

d. Participation in professional growth activities 
brings increased responsibility, status, or 
recognition to teacher. 



Strongly 


Somewhat 


Somewhat 


Strongly 


Agree 


Agree 


Disagree 


Disagree 


□ 


□ 


□ 


□ 


□ 


□ 


c 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 



Data on the financing of professional development activities (Taxonomy element IVB) can 
also be obtained from administrators. 



(PX4) Please answer the following questions about financing professional development in this school. 

a. Is professional development an identifiable line item in your budget? 

C No (Go to part b) 

D Yes 

If yes, 

• What percentage does it represent of the total budget? 

• What percentage of the total professional development budget comes 

from non-school district sources, such as special grants or federal funds? 

b. Does completion of sufficient professional development lead to increases in teacher salary? 

G Yes 

□ No 

c. Are teachers reimbursed for their expenses incurred attending professional development activities (e.g., tuition, 
conference fees, workshop fees)? 

□ Yes 

□ No 



We also recommend maintaining item P25e on teacher influence in determining the content of in- 
service programs. 
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(T33) Check all the types of support that have you received during the current school year for in-service education or 
professional development in your main teaching assignment field. 

□ Released time from teaching {i.e., when your regular teaching responsibilities are covered by someone else ) 

□ Scheduled time (i.e., time built into your schedule for professional development) 

□ Sabbatical or leave 

□ Travel and/or per diem expenses 

□ Tuition and/or fees 

□ Professional growth credits 

□ None of the above 



SASS 93-94 



Similarly, adding an additional query to item 47 can help researchers understand the 
functio nin g of school culture (Taxonomy element IVC) in supporting professional development efforts 
of individual teachers. 



(T47) Do you agree or disagree with each of the following statements? 



Strongly Somewhat Somewhat Strongly 

Agree Agree Disagree Disagree 

I have time outside of my regular teaching during 

the regular school week to work with my peers on o o \Z 3 

curriculum and instruction . 



In addition, two new items would help provide supplementary insights as to whether school 
and district policies reinforce the importance of professional development and would gather needed 
info rma tion about the financial aspects of professional development. School administrators could 
answer the following question about administrative incentives to provide adequate time and support 
for teachers’ professional development (Taxonomy element IV A): 



professional development content produced from interviews would not represent a significant 
improvement over those obtained from surveys. 



Observations 



Observational data from specific professional development activities can better document 
details of the enacted content of professional development and lead to a more accurate understanding 
of content than that provided by survey responses. Such observational data can move analysts beyond 
interpreting only intended lesson plans to analyzing observational data with which to better understand 
the enacted curriculum of professional development. Observing teacher practice would not provide 
substantive information on the content of professional development programs. 



Student Performance 



Collecting data on student performance would not inform our understanding of the content of 
professional development. 



Measuring Context 

To effect sustained change, professional development has to be supported at key junctures by 
institutional practices, financed reliably and adequately (as measured against some standard), and 
integrated into social expectations so that teacher learning activities are pan of the taken-for-granted 
realities of school life. 



Surveys 



SASS. Several items in the 1993-94 SASS address the issues of institutional support, 
financing, and school culture, but judicious modification could improve and expand the information 
collected without interrupting existing time series data. For example, in item T33 collecting data on 
institutional support for professional development (Taxonomy element IV A), the parenthetical 
expression further explains the intention of the item; the third item fills a gap in the original question. 



S ummar y of Specific Recommendations for SASS Items 



We recommend revising five current items on the SASS Teacher Questionnaire (items 30, 3 1 , 
32, 33, 47), and adding three items to the Teacher Questionnaire and four items to the Principal 
Questionnaire. We also recommend combining two current items (T31/T32), maintaining two items 
as they are (T44b, P25e), and eliminating one item (T63). Finally, we recommend investigating 
analysis of student outcome data for one element (Taxonomy element VD). We summarize here our 
recommendations for revisions and additions to SASS according to the taxonomy elements they 
support. 

Revise Add Maintain Eliminate 

I. Design 



A. Connects with goal 


T30 




B. Specific elements 


T47 


PX1 and PX2 T44b 


Delivery 






A. Range of options 




T30 


B. Specific types 


T30 




C. Duration and intensity 


T30 




D. Community resources 




TX1 


Content 






A. Subject matter 


T31/T32 


TX2 


B. Instructional strategies 


T31/T32 




C. Organizational skills 


T31/T32 




Context 






A. Institutional report 


T33 


PX3 


B. Financing 




PX4 


C. School culture 


T47 


P25e 


Outcomes 






A. Teachers’ knowledge 


T31/T32 




B. Instructional practice 


T31/T32 


TX3 


C. School changes 


T30 




D. Student changes 




Other 
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VII. Conclusion 



In its simplest conception, student learning is a product of sustained work on substantively 
sound academic tasks appropriately matched to the student’s needs. Future refinements or more 
elaborated notions of what it really means to know something or know how to do something may 
change the terms of discourse about student learning, but will not alter the need to keep the quality 
and quantity of student engagement and effort as the focus of improvement efforts. Teaching quality 
influences student work in very direct ways: through the degree of substantive rigor and 
appropriateness of academic tasks, through task structures that engage students, through the classroom 
climate that governs the extension of work time within the boundaries of schooltime, and through 
appropriate integrated assessment systems that in themselves promote improved student learning. 
Professional development, properly conceived and realized, is the obvious process through which our 
experienced teacher corps will incorporate new understandings of student learning into their practice 
and incoming teachers will sustain their own professional learning curve. 

As new understanding of the processes of learning has informed assumptions about good 
teaching for children, it has also stimulated new approaches to professional development. These 
approaches provide for teachers to construct professional knowledge, participate in identification of 
their learning needs, and share the work of creating organizational arrangements that support their 
teaching more effectively. While some may argue that the lack of high correlation between 
investments in professional development and student achievement outcomes offers little support for 
diverting scarce resources to professional development, many are now coming to understand that 
investments in teacher learning could pay high dividends in student achievement. Limitations of 
present measurement capacity do not dim the logic of the assumptions that teaching quality contributes 
to student learning, that teaching quality is itself the product of teacher learning experiences, and 
thoughtful, well-planned professional development activities can be significant teacher learning 
experiences. 

The immediate goal of professional development is to improve the quality of teaching, striving 
to maximize teachers’ skills in assessment, classroom management, pedagogy, and specific forms of 
content expertise. And the ultimate objective of high-quality teaching is to maximiz e student learning 
by matching the task to the learner, providing substantive and appropriate work, and creating 
sufficient academic learning time for the task to be successfully completed. 

Similarly, the ultimate goal of data collection about professional development ought to be to 
provide the basis for charting and understanding the current state of the art, its prevalence in the 
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nation’s schools, and its change over time. To do this, we must identify and agree on the elements 
that are important, measurable, and representative of the professional development process. The 
taxonomy on pages 25 through 29 provides a basis for that judgment. Research on professional 
development to date suggests that adding to SASS the items suggested, initiating periodic 
supplementary surveys to enrich the picture of particular aspects of professional development, and 
occasionally using more labor-intensive data collection strategies to validate other methods and expand 
the data pool will improve the adequacy and utility of the information used to develop and implement 
federal education policy. 
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Appendix A 



The National Education Goals 
Goal 4 

Teacher Education and Professional Development 



By the year 2000, the Nation's teaching force will have access to programs for the continued 
improvement of their professional skills and the opportunity to acquire the knowledge and skills 
needed to instruct and prepare all American students for the next century. 

Objectives: 

• All teachers will have access to preservice teacher education and continuing 
professional development activities that will provide such teachers with the knowledge 
and skills needed to teach to an increasingly diverse student population with a variety 
of education, social, and health needs. 

• All teachers will have continuing opportunities to acquire additional knowledge and 
skills needed to teach challenging subject matter and to use emerging new methods, 
forms of assessment, and technologies. 

• States and school districts will create integrated strategies to attract, recruit, prepare, 
retrain, and support the continued professional development of teachers, 
administrators, and other educators, so that there is a highly talented work force of 
professional educators to teach challenging subject matter. 

• Partnerships will be established, whenever possible, among local educational agencies, 
institutions of higher education, parents, and local labor, business, and professional 
associations to provide and support programs for the professional development of 
educators. 



National Education Goals Panel, 1994 



Appendix B 



U.S. Department of Education 
Professional Development Principles 



Principles of High-Quality Professional Development 

The mission of professional development is to prepare and support educators to help all students 
achieve to high standards of learning and development. 

Professional Development: 

focuses on teachers as central to student learning; yet includes all other members of the 
school 

focuses on individual, collegial, and organizational improvement; 

respects and nurtures the intellectual and leadership capacity of teachers, principals, and 

others in the school community; 

reflects best available research and practice in teaching, learning, and leadership; 

enables teachers to develop further expertise in subject content, teaching strategies, uses of 

technologies, and other essential elements in teaching to high standards; 

promotes continuous inquiry and improvement embedded in the daily life of schools; 

is planned collaboratively by those who will participate in and facilitate that development; 

requires substantial time and other resources; 

is driven by a coherent long-term plan; 

is evaluated ultimately on the basis of its impact on teacher effectiveness and student 
learning; and this assessment guides subsequent professional development efforts 



U.S. Department of Education, 1996 



Appendix C 



American Federation of Teachers 
Professional Development Guidelines 

S ummar y 

Professional development is a continuous process of individual and collective examination of practice. 
It should empower individual educators and communities of educators to make complex decisions, 
identify and solve problems, and connect theory, practice, and student outcomes. It should also 
enable teachers to offer students the learning opportunities that will prepare them to meet world-class 
standards in given content areas and to successfully assume adult responsibilities for citizenship and 
work. 



1 . Professional development should ensure depth of content knowledge. 

2. Professional development should provide a strong foundation in the pedagogy of 
particular disciplines. 

3. In addition to content and pedagogical content knowledge, professional development 
should provide more general knowledge about the teaching and learning processes and 
about schools as institutions. 

4. Effective professional development is rooted in and reflects the best available 
research. 

5. Professional development should contribute to measurable improvement in student 
achievement. 

6. Effective professional development expects teachers to be intellectually engaged with 
ideas and resources. 

7. Effective professional development provides sufficient time, support, and resources to 
enable teachers to master new content and pedagogy and to integrate these into their 
practice. 

8. Professional development should be designed by representatives of those who 
participate in it, in cooperation with experts in the field. 

9. Professional development ought to take a variety of forms, including some we have 
not typically considered. 



American Federation of Teachers, 1995 
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Professional Development Survey Items 



Appendix D-l 

Abbreviated Taxonomy of Professional Development Indicators 



Design 

A. Connection with broader school improvement goals 

B. Design elements 

Delivery 

A. Offers a range of roles and responsibilities with multiple ways to participate 

B. Provides for specific types of professional development activity 

C. Provides activities of sufficient duration and intensity to meet goals 

D. Integrates professional community resources effectively 

Content 

A. Subject matter 

B. Instructional Strategies 

C. Interpersonal/Organizational Skills 

Context 

A. Institutional Support 

B. Financing 

C. School Culture 

Outcomes 

A. Teachers’ knowledge/attitudes 

B. Teachers’ classroom practice 

C. School level changes 



D. 



Student level changes 



Appendix D-2 

Surveys with Professional Development Items 



American Public School Teacher Survey: National Education Association, 1991 

California Learning Assessment System: Teacher Survey, 1994 

Classroom Effects of Reform: Chicago Panel on School Policy, Teacher Surveys, 1994 

Computers in Education Survey: International Education Association (EE A), 1992 

Connecticut State Department of Education: Teacher Survey, 1994 

Instructional Practices and Content Survey: Policy Studies Associates, 1995 

Kentucky Title I Reading Teachers Survey: Muchmore, 1994 

Local Systemic Change Teacher Questionnaire: Horizon Research Inc., 1995 

National Assessment of Educational Progress: Reading, Math, and Social Studies Teacher Surveys, 
1991-92 

National Education Longitudinal Study, Teacher Surveys, Base Year, First Followup and Second 
Followup: National Center for Education Statistics, 1988, 1990, 1992 

National Longitudinal Study of Youth: Bureau of Labor Statistics, 1994 

National Survey of Science and Mathematics Education: Horizon Research Inc., Math Teacher 
Questionnaire, 1993 

Pilot Teacher Survey: SRI, International, 1994 

Reform Up Close: Consortium for Policy Research in Education (CPRE), University of Wisconsin, 

1994 

School Based Management Study: CPRE Teacher Surveys, 1993 

Schools and Staffing Survey: National Center for Education Statistics, 1990-91, 1993-94 

Standards-Based Professional Development Program: Policy Studies Associates, 1995 

State Collaborative on Assessment and Student Standards: Council of Chief State School Officers, 

1995 

Survey of the Conditions of Teaching and Learning: National Education Association, 1991-1992 

Survey of Elementary Mathematics Education in California: Center for Research on the Context of 
Teaching, Teacher Questionnaire, 1994 

Third International Mathematics and Science Study: IEA, Teacher Questionnaires, 1995 
Upgrading Mathematics Study: CPRE, Teacher Survey, 1992 
Validating National Curriculum Indicators: RAND, 1994 
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Existing Professional Development Survey Items 
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I. Design 

II. Delivery 

III. Content 

IV. Context 

V. Outcomes 



I. Design 

A. Connection With Broader School Improvement Goals 

(19) Now indicate your opinion about how well each statement applies to your school and 
district. (Circle one for each statement.) 







Strongly 








Strongly 


Don't 






Disagree 








Agree 


Know 


a. 


The principal of this school is well-informed 
about [standards and curriculum guides] 


I 


2 


3 


4 


5 


9 


b. 


There is a school-wide effort to achieve the 
kind of mathematics education promoted by 
[standards and curriculum guides]. 


1 


2 


3 


4 


5 


9 


c. 


Our district is providing extensive staff 
development based on or more of [standards 
and curriculum guides]. 


1 


2 


3 


4 


5 


9 



Center for Research on the Context of Teaching 



(20) Which of the following does your school make available to you? (Circle one on each 
line.) 





No 


Yes, but 
I do not 
use 


Yes, and 
I use 


Don't 

Know 


b. Workshops/in-services related to new 
mathematics standards 

c. Release time to attend off-site workshops or 


1 


2 


3 


9 


conferences related to new mathematics 
standards 


1 


2 


3 


9 



Center for Research on the Context of Teaching 



Please rate the extent to which the Then indicate what impact 
following have changed in the past Chicago School Reform has had 
three years on each 





Worse 


No 

Change 


Better 


Negative 


None Positive 


Professional growth opportunities 


□ 


□ 


n 


□ 


□ C 






l 






Classroom Effects of Reform 
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(1) To what extent do you agree with the following statements about your school? (Circle 
the appropriate number, using the following scale: 1 = Strongly agree, 2= Tend to agree, 
3=Tend to disagree, 4=Strongly disagree.) 



n. Staff development activities relate to 

my school’s goals or needs 12 3 4 

Conditions of Teaching and Learning 



B. Design Elements 

(T44) At this school, how much actual influence do you think teachers have over school 
policy in each of the following areas? 



No 

influence 



A great deal of 
influence 



b. determining the content of inservice 

programs <)□ lD 2D sD 4^ sD 

Schools and Staffing Survey 93-94 



Variations appeared in: RUC (32) and CER (20). 



(41) Please indicate which of the following statements are true of your participation in staff 
development activities during the [past] year. 



Yes No 



a. I was required to participate in staff development activities prior to the start of academic year. 1 0 

b. I have participated in staff development activities on a regular basis throughout the academic year. 1 0 

c. I have participated in staff development activities a few times during the academic year. 1 0 

d. I have participated in no staff development activities during the academic year. 1 0 



National Longitudinal Study of Youth 



I. Design 
n. Delivery 
HI. Content 

IV. Context 

V. Outcomes 



ft 



ft 



ft 



(8) Generally speaking, to what extent has the following been true of science-related 
professional development you have experienced in your district in the last year? 











A 


To a 






Not at 


Just a 


A fair 


substantial 


great 


Don't 


I am involved in planning my science- 


all 


little 


amount 


amount 


extent 


know 


related professional development. 


iD 




3 □ 


4 ecu 


sG 


«□ 


Teachers in the district are involved in 


planning for professional 
development. 

Professional development activities 


iG 


2 CD 


3 D 


4O 


sD 


CTv 

□ 


draw on expertise of teachers in the 
district. 


iG 


2O 


3 D 


4 D 


sG 


6 G 



Local Systemic Change Teacher Questionnaire 



(12) In your opinion, how much influence do employees— other than administrators— have on 
^ decisions made in your school in the following areas? (Circle the appropriate number.) 





Great 


Moderate 


No 




Influence 


Influence 


Influence 


Training opportunities 


1 


2 


3 








Conditions of Teaching and Learning 



ft 
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V. Outcomes 



Delivery 

A. Offers a Range of Roles and Responsibilities With Multiple Wavs to Participate 

(T30) Since the end of last school year, in which of these activities related to teaching have 
you participated? Mark (X) to all that apply. 

iD School District sponsored workshops or in-service programs 
2 O School Sponsored workshops or in service programs 

3 [H University extension or adult education courses 

4 □ College courses in your subject field 

sD Professional growth activities sponsored by professional associations 
sD Committee to integrate academic skills into the vocational curriculum 

?□ Other curriculum committee 

sD Committee on selecting textbooks or materials 

oD None of the above 

Schools and Staffing Survey 93-94 

Variations appeared in: PTS (21 and 23); CER (19); CRCT (3); SCASS (17-23); NSSME 
(11); NELS First Followup (20); and CLAS (7). 



I. Design 

II. Delivery 

III. Content 

IV. Context 

V. Outcomes 



(12) What professional development experiences did you have in 1994-95? 



a. I attended a slate or national education conference 

b. I attended a workshop on curriculum/ instruction 

c. I attended a workshop on school organizations and management that 

was sponsored by the district or my school 

d. I took a graduate course through a college or university 

e. I served on a school based management committee 

f. I read an educational journal on a regular basis 

g. I met with colleagues on a regular basis to review and analyze student work . . . 

h. I met with colleagues to develop curriculum, lessons, or assessments 

i. I made a presentation at a conference or other professional meeting 



(13) Of these professional development experiences, which one was the most beneficial for 
you? 



(14) What kinds of professional development activities would you like to participate in the 
coming year? 

a. Attend a state or national education conference 

b. Attend a school- or district-sponsored workshop on curriculu m/instruction .... 

c. Attend a school- or district-sponsored workshop on school organizations 

and management 

d. Take a graduate course through a college or university 

e. Serve on a school-based management committee 

f. Read an educational journal on a regular basis 

g. Meet with colleagues on a regular basis to review and analyze student work . . . 

h. Meet with colleagues to develop curriculum, lessons, or assessments 

i. Make a presentation at a conference or other professional meeting 



Standards-Based Professional Development Program 
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(43) Below are listed several types of professional growth activities. Please indicate if you 
have participated in any of these activities during the past three years. 



a. 


Sabbatical leave: 

Full-time study at a college or university 


This Year 
1990-1991 

□ 


Last Year 
1989-1990 

□ 


Year Before 
Last 

1988-1989 

□ 




Travel 


□ 


□ 


□ 




Other 


□ 


□ 


ri 


b. 


Other educational travel 


□ 


□ 


□ 


c. 


System-sponsored workshops during the school year 


□ 


□ 


□ 


d. 


System-sponsored workshops during the summer 


n 


p-t 


n 


e. 


Work on curriculum committee 


□ 


□ 


□ 


f. 


Committee work or special assignment (other than 


□ 


□ 


G 


o 

O # 


curriculum) 

University extension courses 


□ 


□ 


□ 


h. 


College courses in education during school year 


D 


□ 


n 


i. 


College courses in subject fields other than education 




□ 


LJ 


j- 


during the school year 

College courses in education during the summer 


□ 


n 


□ 


k. 


College courses in subjects other than education 


□ 


D 


; j 


i. 


during the summer 

Professional growth activities sponsored by 


□ 


□ 


□ 


m. 


professional association(s) 
Educational TV 


□ 


D 


□ 


n. 


Exchange teaching, domestic 


n 


□ 


□ 


0 . 


Exchange teaching, foreign 


□ 


□ 


□ 


P- 


International education meetings 


□ 


□ 


□ 



American Public School Teacher Survey’ 
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HI. Content 
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V. Outcomes 



B. Provides for Specific Types of Professional Development Activity 

(8) Generally speaking, to what extent has the following been true of science-related 
professional development you have experienced in your district in the last year? 













A 


To a 








Not at 


Just a 


A fair 


substantial 


great 


Don't 






all 


little 


amount 


amount 


extent 


know 


f. 


I am given time to reflea on my 
learning and how to apply it to the 


iD 


iD 


3 □ 




sD 


eU 




classroom. 














o. 


I am given time to work with other 
teachers as part of my professional 
development. 


iD 


iD 


3 □ 


4 □ 


sD 


□ 

MD 


i. 


Professional development includes 
activities other than training 
workshops (e.g.. study groups, peer 


iD 


iD 


3 □ 


4 □ 


sD 


«□ 



coaching, mentoring). 



Local Systemic Change Teacher Questionnaire 



(12) Approximately how many hours per week do you normally spend on each of the 
following activities outside of the formal school day? 



professional reading and 
development activity 



□ 



less 






more 


than 1 


1 - 2 


3-4 


than 4 


hour 


hours 


hours 


hours 


□ 


□ 


□ 


□ 



(7) How often do you read professional journals? 



^Frequently 

Seldom 



_Occasionally 

Never 



Third International Mathematics and Science Study 



Kentucky Title l Reading Teachers Survey 



_ o 
• ERLC 
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IE. Content 

IV. Context 
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(9) How often do you apply what you read in professional journals to your classroom 
practice? 

Frequently Occasionally 

Seldom Never 



Kentucky Title I Reading Teachers Survey 



(39) Excluding any team teaching partners, how often do you visit another teacher’s 
classroom to observe their teaching? 

Never 

Annually 

Semi-Annually 

Bimonthly 

Monthly 

Weekly 

More than once a week 

Third International Mathematics and Science Study 




(40) Excluding any team teaching partners, how often does another teacher observe your 
teaching to provide you with feedback? 



Never □ 

Annually □ 

Semi-Annually □ 

Bimonthly □ 

Monthly □ 

Weekly □ 

More than once a week □ 



Third International Mathematics and Science Study 



A variation appeared in PTS (17 and 18). 



8 
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» 



i 



» 



» 



(38) Since the beginning of the school year, how many times has your teaching been 
observed for purposes other than formal evaluation? 



Never i 

Once 2 

Twice 3 

3-4 times 4 

5-9 times 5 

10 or more times 6 



Reform Up Close 

Variations appeared in: CRCT (37) and NSSME (11). 

This school year how often have you: 

3-4 5-9 10 Or 





Never 


Once 


Twice 


Times 


Times 


More 


Had colleagues observe your classroom 


□ 


□ 


□ 


□ 


□ 


□ 


Visited other teachers' classrooms 


□ 


□ 


□ 


□ 


□ 


□ 


Received meaningful feedback on your 


□ 


□ 


□ 


□ 


□ 


□ 


performance from colleagues 

Received useful suggestions for curriculum 


□ 


□ 


□ 


□ 


□ 


□ 


materials from colleagues 

Invited someone in to help teach your class 


□ 


□ 


□ 


□ 


□ 


□ 



Classroom Effects of Reform 



(37) This question concerns how teachers interact in your school. Please indicate about how 
many teachers in your school do each of the following. (Circle one for each statement.) 



Work together to develop curriculum and 

instructional materials 

Observe each other teaching 

Offer advice or help to each other 

Share ideas on teaching 

Promote innovative teaching practices 



» 



No 

teachers 


Some 

teachers 


Most 

teachers 


Ail 

teachers 


Don't 

Know 


1 


2 


3 


4 


9 


1 


2 


3 


4 


9 


1 


2 


3 


4 


9 


1 


2 


3 


4 


9 


1 


2 


3 


4 


9 



Center for Research on the Context of Teaching 




9 
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(25) Please indicate the approximate number of hours you have spent on each of the following 
types of professional development in science/science education in the past 12 months (Do 
not count preservice preparation.) 

Number of Hours 





0 


1-6 


7-15 


16-35 


36-50 


51-75 


76-100 


100+ 


Structured Professional Development 
(approximate total) 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


Di stria-level activities (summer 
institutes, seminars) 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


School-level activities (workshops or 
study groups) 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


Individual activities (coaching, formal 
action research projects) 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


Unstructured Professional Development 
(approximate total) 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


Group activities (informal discussions 
with teachers about teaching/leaming) 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


Individual activities (professional 
reading) 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 



(26) Please indicate the approximate number of hours you have spent on each of the following 
types of professional development in science/science education in the past five years . (Do not 
count preservice preparation.) 



Number of Hours 





0 


1-6 


7-15 


16-35 


36-50 


51-75 


76-100 


100 + 


Structured Professional Development 
(approximate total) 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


District-level activities (summer 
institutes, seminars) 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


School-level activities (workshops or 
study groups) 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


Individual activities (coaching, formal 
action research projects) 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


Unstructured Professional Development 
(approximate total) 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


Group activities (informal discussions 
with teachers about teaching/leaming) 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


Individual activities (professional 
reading) 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 



Local Systemic Change Teacher Questionnaire 



o 
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(52) Which of the following topics have you learned about primarily in formal training 
experiences and which have you learned primarily on your own? 



a. How to integrate software into existing math 
lessons 

b. How to organize class activities to allow for 
computer use during class time 

c. How to write programs in a computer programming 
language 

d. Use of word-processing programs 

e. Use of other computer applications 



Formal 


Learned 


Have not 


training 


on own 


learned 


Formal 


Learned 


Have not 


training 


on own 


learned 


Formal 


Learned 


Have not 


training 


on own 


learned 


Formal 


Learned 


Have not 


training 


on own 


learned 


Formal 


Learned 


Have not 


training 


on own 


learned 



Computers in Education Survey (Math Teachers) 



(37) Since the beginning of this school year, how much time per month have you spent 
meeting informally with other teachers on lesson planning, curriculum development, or other 
instructional matters? 



Less than 15 minutes 1 

15-29 minutes 2 

30-59 minutes 3 

1 hour or more, less than 5 4 

5 hours or more, less than 10 5 

10 hours or more 6 



Reform Up Close 

Variations of the above item ask teachers about collaboration and time spent formally or 
informally with other teachers on teaching approaches, guidance, or other instructional 
matters. These include: UGM (12); NELS Base Year (30), First Followup (23); TTMSS (10 
and 13); and PTS (16). 



11 
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This school year, how often have you had conversations with colleagues about: 





Less Than 


2 or 3 Times 


Once or Twice 


Almost 




Once A Month 


A Month 


A Week 


Daily 


What helps students learn best 


□ 


□ 


□ 


□ 


Managing classroom behavior 


□ 


□ 


□ 


! i 


Development of new curriculum 


□ 


□ 


□ 


G 


The goals of this school 


□ 


□ 


□ 


S-J 



Classroom Effects of Reform 



(6) Did you collaborate with anyone outside your school on curriculum, instruction, 
assessment, or student achievement in 1994-95? 



a. Yes, I collaborated with someone at another school in my school" s feeder 

pattern 

b. Yes, I collaborated with someone at a college or university 

c. Yes, I collaborated with a community member 

d. Yes, I collaborated with (SPECIFY): 

e. No, I did not collaborate with anyone outside of my school in 1994-95 . . 



Standards-Based Professional Development Program 




12 



ICO 



I. Design 

II. Delivery 
ID. Content 
TV. Context 
V. Outcomes 



(24a) In addition to the college courses you have already reported, have you ever participated 
in any teacher workshops or in-service training which included 30 hours or more of class 
attendance? 

□ Yes - (continue) 

□ No 

(b) Was this tr aining relevant to your current MAIN teaching assignment field? 

□ Yes 

□ No 



(c) What was your MAJOR purpose for taking this training? (Mark only one box.) 

□ To fulfill initial certification requirements for current position 

□ To maintain and/or improve qualifications for current position (including meeting recertification requirements) 

□ To train to teach a different subject matter area 

□ To train to teach at a different grade level 

□ To train to teach handicapped students or students with learning disabilities 

□ To train to teach other special student populations (e.g., Native American, limited English proficiency, etc.) 

□ To acquire credentials in new nonteaching areas (e.g., administration, guidance counseling) 

□ Other (Describe): 

Schools and Staffing Survey 90-91 

C. Provides Activities of Sufficient Duration and Intensity to Meet Goals 

(8) GeneraUy speaking, to what extent has the following been true of science-related 
professional development you have experienced in your district in the last year? 











A 


To a 






Not at 


Just a 


A fair 


substantial 


great 


Don't 




all 


little 


amount 


amount 


extent 


know 


I receive follow-up support as I try to 
implement what I learn in professional 
development activities. 


iD 


iD 


3 D 


4D 


sD 


6D 


Professional development programs 
are sustained over time, with ample 


iD 


iD 


3 □ 


4D 


sD 


6D 



follow-up activities and experiences. 

Local Systemic Change Teacher Questionnaire 
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(38) How much do you agree or disagree with each of the following statements? 



38. I am provided with satisfactory staff 
development training in new teaching 
methods. 



Strongly 

Agree 

a 



Somewhat 

Agree 

a 



Somewhat 

Disagree 

a 



Strongly 

Disagree 

□ 



Connecticut State Department of Education 



D. Integrates Professional Community Resources Effectively 
We found no items to address this aspect of delivery. 



HI. Content 

A. Subject Matter 

(T31) Since the end of last school year, have you participated in any in-service or 
professional development programs which focused on the following topics? 

b. Methods of teaching your subject or field 

□ Yes - How many hours did the program last? — □ 8 hours or less 

□ No □ 9-32 hours 

□ More than 32 hours 



c. In-depth study in your subject field 

□ Yes - How many hours did the program last? — □ 8 hours or less 

□ No □ 9-32 hours 

□ More than 32 hours 

Schools and Staffing Survey 93-94 
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(1-7) What level of exposure, if any, have you had to each of the following topics or areas? 



1. Methods of teaching 


One or more college 
or university courses 


Part of a college or 
university course 


In-service 

training 


Little or i 
exposure 


elementary mathematics 
2. Number systems and 


0 


0 


0 


0 


numeration 
3. Measurement in 


0 


0 


0 


0 


mathematics 


0 


0 


0 


0 


4. Geometry 


0 


0 


0 


0 


5. Probability/ statistics 


0 


0 


0 


0 


6. Abstract/ linear algebra 


0 


0 


0 


0 


7. Calculus 


0 


0 


0 


0 



National Assessment of Educational Progress Math Teacher Survey 



A variation appeared in the NAEP Reading Teacher Survey (1-7). 



(35) Since the beginning of this school year, how many half-days have you spent in fo rmal 
inservice programs related directly to improving math/science curriculum and instruction? 



None 1 

1-2 half-days 2 

3-4 half-days 3 

5-6 haif-days 4 

7-8 half-days 5 

More than 8 half-days 6 



Reform Up Close 



Variations appeared in: UGM (10); SCASS (15, 16); PTS (23); NAEP 92 Reading/Math (14, 
15); NAEP 92 Social Studies (10); NSSME 93 (10); and CRCT (2). 
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B. Instructional Strategies 0 

(T31) Since the end of last school year, have you participated in any in-service or 
professional development programs which focused on the following topics? 



Uses of educational technology for instruction (e.g., use of computer, satellite learning) 

□ Yes - How many hours did the program last? — □ 8 hours or less 

□ No □ 9-32 hours 

□ More than 32 hours 

Student assessment (e.g, methods of testing, evaluation, performance assessment) 



□ Yes - How many hours did the program last? — □ 8 hours or less 



□ No 



□ 9-32 hours 

□ More than 32 hours 



Cooperative learning in the classroom 



□ Yes - How many hours did the program last? -* □ 8 hours or less 



□ No 



□ 9-32 hours 

□ More than 32 hours 

Schools and Staffing Survey 93-94 



(T63a) Have you received any training for teaching limi ted English proficient (LEP) 
students? 

□ Yes 

□ No 

Schools and Staffing Survey 93-94 



(8-14) Have you ever had training in any of the following either in college courses or in in- 4 

service education? 







Yes 


No 


8. 


Estimation 


[A] 


[B] 


9. 


Problem solving in mathematics 


[A] 


[B] 


10. 


Use of manipuiatives 


[A] 


PI 


11. 


Use of calculators in mathematics instruction 


[A] 


PI 


12. 


Understanding students' thinking about mathematics 


[A] 


PI 


13. 


Gender issues in the teaching of mathematics 


[A1 


PI 


14. 


Teaching students from different cultural backgrounds 


[A1 


PI 



National Assessment of Educational Progress Math Teacher Survey 
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(1-14) During the past five years, have you ever had training in any of the following, either in 
college courses or in in-service education? Fill in one oval on each line. 







Yes 


No 


1 . 


Study strategies 


[A] 


[B] 


2. 


Motivational strategies 


[A] 


[B] 


3. 


Teaching critical thinking 


[A] 


[B] 


4. 


The role of students’ prior 








knowledge in their reading 


[A] 


[B] 


5. 


Diagnosis and remediation of reading 








difficulties 


[A] 


(B] 


6. 


Ability grouping 


[A] 


(B] 


7. 


Literature-based reading instruction 


[A] 


[B] 


8. 


Reading assessment 


[A] 


[B] 


9. 


Content area reading 


[A] 


(B1 


10. 


Combining reading and writing 


[A] 


[B] 


n. 


The whole language approach to 








teaching reading 


[A] 


(B1 


12. 


Phonics in the teaching of reading 


[A] 


[B] 


13. 


Individual reading programs 


(A) 


(B1 


14. 


Teaching students from different 








cultural backgrounds 


(A] 


[B] 



National Assessment of Educational Progress Reading Teacher Survey 



(8) Generally speaking, to what extent has the following been true of science-related 
professional development you have experienced in your district in the last year? 





Not at 


Just a 




all 


little 


Professional development experiences 


emphasize inquiry in science and 
science teaching. 


iU 


2 O 





A 


To a 




A fair 


substantial 


great 


Don’t 


amount 


amount 


extent 


know 


3 D 


4O 


sD 


6 n 



Local Systemic Change Teacher Questionnaire 
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C. Interpersonal/Organizational Skills 

(16) How often is the following training offered in your school? 

Several 



a. 


Assertiveness training 


Monthly or 
More Often 
1 


Times 
A Year 
2 


Once 
A Year 
3 


Every Few 
Years 
4 


Not 

Offered 

5 


c. 


Brainstorming techniques 


1 


2 


3 


4 


5 


e. 


Communication skills 


1 


2 


3 


4 


5 


o. 


Conflict resolution 


1 


2 


3 


4 


5 


h. 


Consensus building 


1 


2 


3 


4 


5 


m. 


Decisionmaking skills 


1 


2 


3 


4 


5 


0. 


Leadership skills 


1 


2 


3 


4 


5 


s. 


Problem solving skills 


1 


2 


3 


4 


5 


u. 


Teamwork training 


1 


2 


3 


4 


5 



National Education Association 



IV. Context 

A. Institutional Support 

(T33) What types of support have you received during the current school year for in-service 
education or professional development in your main teaching assignment field? 

□ Released time from teaching 

□ Scheduled time (i.e.. time built into your schedule for professional development) 

□ Travel and/or per diem expenses 

□ Tuition and/or fees 

□ Professional growth credits 

□ None of the above 



Schools and Staffing Survey 93-94 

Variations appeared in: NELS (19); UGM (11); RUC (36); and CRCT (20). 
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I. Design 

II. Delivery 
in. Content 

IV. Context 

V. Outcomes 



(8) Generally speaking, to what extent has the following been true of science-reiated 
professional development you have experienced in your district in the last year? 





Not at 


Just a 




all 


little 


Adequate opportunities are available 
to me for science-related professional 
development. 

I am provided with incentives for 


iU 


iD 


participation (e.g., released time, 
substitute pay, renewal credit, 


iD 


iD 


stipends). 

I am provided with opportunities to 
try out new teaching strategies. 


.□ 


id 



A fair 
amount 


A 

substantial 

amount 


To a 
great 
extent 


Don't 

know 


3 □ 


4 □ 


sd 


«□ 


3 □ 


4 □ 


sD 


6 CD 


3 □ 


4 q 


sd 


ed 



Local Systerruc Change Teacher Questionnaire 



(42) Please provide your opinion about each of the following statements. 





Strongly 

Disagree 


Disagree 


No 

Opinion 


Agree 


Strongly 

Agree 


r. I have time during the regular school 

week to work with my peers on science 
curriculum and instruction. 


1 


2 


3 


4 


5 



Pilot Teacher Survey 

Variations appeared in: NSSME (lr) and SCASS (56). 



(38g) How well does (this statement) describe conditions in your school? 

Strongly Strongly 

Disagree Agree 

g. A specialist in mathematics education 1 2 3 4 5 

regularly works with teachers in this school. 

Center for Research on the Context of Teaching 
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I. Design 
13. Delivery 
HI. Content 

IV. Context 

V. Outcomes 



(17) How would you rate your personal satisfaction with each area below? (Circle the 
appropriate number.) 



Very Very 

Satisfied Dissatisfied 

c. Your learning opportunities 12 3 4 



Conditions of Teaching and Learning 



B. Financing 

We found no items to address this aspect of context. 



C. School Culture 

(T47) Do you agree or disagree with each of the following statements? 





Strongly 


Somewhat 


Somewhat 


Strongly 




Agree 


Agree 


Disagree 


Disagree 


f. Teachers participate in making most of the 


important educational decisions in this school. 


c 


D 


D 


0 


p. There is a great deal of cooperative effort 


among the staff members. 


2 


□ 


□ 


□ 








Schools and Staffing Survey 93-94 



Variations appeared in: PTS (32); RUC (30); and NELS First Followup (2). 
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I. Design 

II. Delivery 
HI. Content 

IV. Context 

V. Outcomes 



ft 



ft 



ft 



ft 



ft 



ft 



(T47) Do you agree or disagree with each of the following statements? 



Strongly 


Somewhat 


Somewhat 


Strongly 


Agree 


Agree 


Disagree 


Disagree 


I make a conscious effort to coordinate the 

content of my courses with that of other □ 

teachers. 


□ 


□ 


□ 



Schools and Staffing Survey 93-94 

Variations appeared in: PTS (32); RUC (18); NELS First Followup (1); and CER (6). 



(P25) Using the scale 0-5, where 0 is none and 5 is a great deal, indicate how much actual 
influence you think each group or person has on decisions concerning the following activities. 



e. Determining content of in-service programs 



State Department of Education 

School District Staff 

School Board 

Principal 

Teachers 

Parent Association 



No 

influence 

oD 


iB 


zB 


oB 


iB 


zB 


oB 


iB 


zB 


oD 


iB 


zB 


oD 


iD 


zB 


oD 


iB 


zB 



3 □ 


*B 


A great deal 
of influence 

sB 


3 □ 


aB 


sB 


3 □ 


4B 


sB 


3 □ 


40 


sB 


zB 


40 


sB 


3 C 


40 


sB 



Schools and Staffing Survey 93-94 
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I. Design 
n. Delivery 

III. Content 

IV. Context 

V. Outcomes 



(8) Generally speaking, to what extent has the following been true of science-related 
professional development you have experienced in your district in the last year? 





Not at 
all 


Just a 
little 


A fair 
amount 


A 

substantial 

amount 


To a 
great 
extern 


Don't 

know 


I am encouraged to develop an 
individual professional development 
plan to address my needs and interests 
related to science education. 


iD 


2O 


3O 


4 D 


sD 


60 



Local Systemic Change Teacher Questionnaire 



(32) To what extent does each of the following statements describe your school? 



l. 


In this school, I am encouraged to experiment 


Strongly 

Agree 


Somewhat 

Agree 


Somewhat 

Disagree 


Strongly 

Disagree 


u. 


with my teaching. 

The department chair of curricular area 


□ 


c 


□ 


- 




coordinator's behavior toward the staff is 
supportive and encouraging. 


0 


- 


a 


- 


V. 


TTie school administration's behavior toward 
the staff is supportive and encouraging. 


□ 


- 


□ 


2 


w. 


Teachers in this school are continually 
learning and seeking new ideas. 


□ 


2 


□ 


2 



Pilot Teacher Survey 



Variations appeared in: NELS First Followup (2) and CER (6). 
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I. Design 

II. Delivery 
HI. Content 

IV. Context 

V. Outcomes 



(19) To what extent does each of the following statements describe your relationships with the 
teachers in your primary subject area in this school? 







Strongly 




No 




Strongly 






Disagree 


Disagree 


Opinion 


Agree 


Agree 


a. 


We share ideas about teaching openly. 


1 


2 


3 


4 


5 


b. 


It is common for us to share samples of 
work done by our students. 


I 


2 


3 


4 


5 


c. 


We regularly meet to discuss particular 
common problems and challenges we are 
facing in the classroom. 


1 


2 


3 


4 


5 


d. 


We often work together to develop 
teaching materials or activities for 
particular classes. 


1 


2 


3 


4 


5 


e. 


We have little idea of each other's 
teaching goals and classroom practices. 


I 


2 


3 


4 


5 


f. 


We often seek each other's advice about 
professional issues and problems. 


1 


2 


3 


4 


5 


a 

o* 


There is a lot of disagreement among us 
about how to teach the subject. 


1 


2 


3 


4 


5 


h. 


We share views of students and how to 
relate to them. 


1 


2 


3 


4 


5 


i. 


Most take a "hands off" attitude toward 
each other's careers. 


I 


2 


3 


4 


5 


j- 


We admire one another's teaching on the 


1 


2 


3 


4 


5 




whole. 










Pilot Teacher Survey 


Please mark the extent to which you agree or disagree with 


each of the following statements. 






Strongly 




No 




Strongly 






Disagree 


Disagree 


Opinion 


Agree 


Agree 




Experienced teachers invite new teachers 
into their rooms to observe, give 
feedback, etc. 

It is common to find a teacher 


I 


2 


3 


4 


5 




demonstrating some new classroom 
practice to colleagues. Teachers design 
instructional programs together. 


1 


2 


3 


4 


5 






Classroom Effects of Reform 



I. Design 

II. Delivery 
HI. Content 

IV. Context 

V. Outcomes 



How many teachers in this school: 

None Some 

Feel responsible to help each other do their best 1 2 

Classroom Effects of Reform 



About Nearly 

Half Most All 

3 4 5 



(42) Please provide your opinion about each of the following statements. 



i. 


I feel supported by colleagues to try out 


Strongly 

Disagree 

1 


Disagree 

2 


No 

Opinion 

3 


Agree 

4 


Strongly 

Agree 

5 


k. 


new ideas in teaching science. 

Science teachers in this school regularly 


1 


2 


3 


4 


5 


1. 


share ideas and materials. 

Science teachers in this school regularly 
observe each other teaching classes as 


1 


2 


3 


4 


5 


P- 


part of sharing and improving 
instructional strategies. 

Most science teachers in this school 
contribute actively to making decisions 


l 


2 


3 


4 


5 



about the science curriculum. 



Pilot Teacher Survey 



Variations appeared in NSSME (1) and CRCT (38). 
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I. Design 
n. Delivery 
EH. Content 

IV. Context 

V. Outcomes 



t V. Outcomes 



» 



» 



» 



» 



A. Teachers’ Knowledge/ Attitudes 

(T32) Please give your opinion about the impact of the professional development programs, 
described in item 3 1 , in which you have participated since last school year. 



Strongly 

Agree 

a. Provided information that was new to me □ 

b. Changed my views on leaching □ 

d. Caused me to seek further information or □ 

training 

e. Were generally a waste of my time □ 





No 




Strongly 


Agree 


Opinion 


Disagree 


Disagree 


□ 


□ 


□ 


D 


□ 


□ 


□ 


□ 


p 


□ 


□ 


□ 


□ 


□ 


□ 


□ 



Schools and Staffing Survey 93-94 



B. Teachers’ Classroom Practice 



(T32) Please give your opinion about the impact of the professional development programs, 
described in item 31, in which you have participated since last school year. 





Strongly 

Agree 


Agree 


No 

Opinion 


Disagree 


Strongly 

Disagree 


Caused me to change my teaching 
practices 


□ 


□ 


□ 


□ 


□ 



Schools and Staffing Survey 93-94 



(25) On a scale of 1 to 5, please rate the overall impact that the SSI programs have had on 
your science/mathematics instruction. (1 = no impact, 5 = large positive impact) 



N/A 

Pilot Teacher Survey 



» 
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BEST COPY AVAILABLE 



I. Design 

II. Delivery 
HI. Content 

IV. Context 

V. Outcomes 



(7) Please rate the effect of each of the following on your science instruction. 





Not a 


A minor 


Somewhat 
of a 


A 

substantial 


A major 


NA/ 

don’t 




problem 


problem 


problem 


problem 


problem 


know 


opportunities for 
professional development 


□ 


□ 


□ 


□ 


□ 


□ 



Local Systemic Change Teacher Questionnaire 

C. School Level Changes 

(29) On a scale of 1 to 5, to what extent has your participation in the SSI program increased 
the involvement of each of the following in your planning and/or teaching science? 

Not at A Great 

All Extent 

a. other teachers 1 2 3 4 5 



Pilot Teacher Survey 



D. Student Level Changes 

We found no items to address this aspect of outcomes. 
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